diff --git a/manage/resources/public/healthchecks-failover.mdx b/manage/resources/public/healthchecks-failover.mdx index b71afb4..cef0844 100644 --- a/manage/resources/public/healthchecks-failover.mdx +++ b/manage/resources/public/healthchecks-failover.mdx @@ -1,144 +1,107 @@ --- -title: "Health Checks" -description: "Configure automated health monitoring and failover for resources" +title: "Health Checks and Failover" +description: "Monitor public resource targets and automatically remove unhealthy targets from routing" --- import PangolinCloudTocCta from "/snippets/pangolin-cloud-toc-cta.mdx"; +Health checks for public resources monitor each target and keep bad targets out of traffic automatically. When a target fails its check, Pangolin marks it unhealthy and removes it from load balancing. When it recovers and passes again, Pangolin adds it back. +## How it works -Pangolin provides automated health checking for targets to ensure traffic is only routed to healthy services. Health checks are essential for building highly available services, as they automatically remove unhealthy targets from traffic routing and load balancing. +For every target on a public resource, Pangolin runs checks at your configured intervals and evaluates the result against your health criteria. -## How Health Checks Work +- Passing checks keep the target in rotation. +- Failing checks remove the target from rotation. +- Recovery checks add the target back after threshold conditions are met. -### Monitoring Process +This gives automatic failover across targets without manual intervention. -Health checks operate continuously in the background: +## Target states -1. **Periodic Checks**: Pangolin sends requests to your target endpoints at configured intervals. -2. **Status Evaluation**: Responses are evaluated against your configured criteria. -3. **Traffic Management**: Healthy targets receive traffic, unhealthy targets are excluded. -4. **Automatic Recovery**: Targets are automatically re-enabled when they become healthy again. +Targets move through three operational states: -## Target Health States +- `Unknown`: initial state before the first check finishes; target may still receive traffic. +- `Healthy`: checks are passing; target is eligible for routing and load balancing. +- `Unhealthy`: checks are failing; target is excluded from routing and load balancing. -Targets can exist in three distinct states that determine how traffic is routed: +## Check types - - - **Initial State**: Targets start in this state before first health check - - **Traffic Behavior**: Unknown targets still route traffic normally - - **Duration**: Until first health check completes - +Public resource target health checks support the same two probe types used by arbitrary health checks: - - **Failed Checks**: Target has failed health check criteria - - **Traffic Behavior**: No traffic is routed to unhealthy targets - - **Load Balancing**: Excluded from load balancing rotation - +- HTTP checks: request a URL and evaluate response behavior (for example status code). +- TCP checks: attempt a TCP connection to a host and port without HTTP semantics. This is useful for non-HTTP services where you only need to verify the port is reachable. - - **Passing Checks**: Target is responding correctly to health checks - - **Traffic Behavior**: Receives traffic according to load balancing rules - - **Load Balancing**: Included in load balancing rotation - - +## Configure health checks on a target -## Configuring Health Checks +1. Open a public resource in the dashboard. +2. In the targets table, open the health check settings for the target. +3. Configure probe parameters and thresholds. +4. Save. - - - In the Pangolin dashboard, navigate to your resource and locate the target in the table. - +Each target can have its own health check settings. - - Click the settings wheel (⚙️) next to the health check endpoint column. - + + Create health check form in the Pangolin dashboard + - - Fill out the health check configuration with your desired parameters. - +## Common parameters - - Save your settings to enable health checking for the target. - - +Some of the most important settings to tune are: -## Health Check Parameters +- `healthy interval`: how often Pangolin probes when a target is currently healthy. +- `unhealthy interval`: how often Pangolin probes when a target is currently unhealthy (usually shorter for faster recovery detection). +- `healthy threshold`: how many consecutive successful checks are required before marking a target healthy again. +- `unhealthy threshold`: how many consecutive failed checks are required before marking a target unhealthy. +- `timeout`: maximum time a probe can take before it is treated as failed. +- HTTP-specific fields: probe scheme (`http`/`https`), path, method, headers, and expected status codes. -### Endpoint Configuration +Use intervals and thresholds together to avoid flapping: short transient blips should not immediately eject a target, and recovery should be confirmed before re-entry. -- **Target Endpoint**: The URL or address to monitor for health status -- **Default Behavior**: Usually the same as your target endpoint -- **Custom Endpoints**: Can monitor different endpoints (e.g., `/health`, `/status`) + +The dashboard includes additional health-check options beyond the examples above. Use this section as a starting point and refer to the full UI field set when configuring production checks. + -### Timing Configuration +## Public resource failover patterns -#### Healthy Interval +### Multi-target redundancy -- **Purpose**: How often to check targets that are currently healthy -- **Typical Range**: 30-60 seconds -- **Consideration**: Less frequent checks reduce overhead +Use multiple targets for the same service. If one goes unhealthy, traffic continues to healthy targets. -#### Unhealthy Interval - -- **Purpose**: How often to check targets that are currently unhealthy -- **Typical Range**: 10-30 seconds -- **Consideration**: More frequent checks enable faster recovery - -### Response Configuration - -#### Timeout Settings - -- **Request Timeout**: Maximum time to wait for a health check response -- **Default Behavior**: Requests exceeding timeout are considered failed -- **Recommended**: Set based on your service's typical response time - -#### HTTP Response Codes - -- **Healthy Codes**: Which HTTP status codes indicate a healthy target -- **Common Settings**: 200, 201, 202, 204 -- **Custom Codes**: Configure based on your service's health endpoint behavior - -## High Availability Strategies - -### Multi-Target Redundancy - -#### Service Redundancy - -Deploy multiple instances of your service across different targets to ensure availability even when some targets fail. - -``` +```text Resource: web-application -├── Target 1: web-01.local:8080 (Site A) - Healthy ✅ -├── Target 2: web-02.local:8080 (Site A) - Unhealthy ❌ -└── Target 3: web-03.local:8080 (Site B) - Healthy ✅ +├── Target 1: web-01.local:8080 (Site A) - Healthy +├── Target 2: web-02.local:8080 (Site A) - Unhealthy +└── Target 3: web-03.local:8080 (Site B) - Healthy Traffic routes to: Target 1 & Target 3 only ``` -### Cross-Site Failover - -#### Geographic Distribution +### Cross-site failover Distribute targets across multiple sites to protect against site-level failures. -``` +```text Resource: api-service ├── Primary Site Targets -│ ├── api-01.primary:8443 - Healthy ✅ -│ └── api-02.primary:8443 - Healthy ✅ +│ ├── api-01.primary:8443 - Healthy +│ └── api-02.primary:8443 - Healthy └── Backup Site Targets - ├── api-01.backup:8443 - Healthy ✅ - └── api-02.backup:8443 - Healthy ✅ + ├── api-01.backup:8443 - Healthy + └── api-02.backup:8443 - Healthy All targets receive traffic via load balancing ``` + +If a whole site fails, only targets from reachable sites continue receiving traffic until health recovers. + +## Related alerting and arbitrary checks + +This page covers health checks attached to public resource targets (available in all editions). + +If you need centralized visibility across checks, standalone non-resource checks, or notifications: + +- See [Alerting health checks](/manage/alerting/health-checks) for org-level health-check visibility and arbitrary health checks. +- See [Alert rules](/manage/alerting/alert-rules) to notify email, webhooks, and integrations when health state changes.