mirror of
https://github.com/prometheus-community/windows_exporter.git
synced 2026-10-18 10:39:05 +02:00
Jan-Otto Kröpke
ae5d0cca2a
The alerts of the mixin now come in one rule group per collector area (windows-exporter, windows-cpu, windows-memory, windows-disk, windows-network, windows-service, windows-system, windows-active-directory, windows-time). Every alert names the collectors it reads and is generated only when _config.collectors enables them, the same switch the dashboard uses; enableActiveDirectory and enableTime set collectors.ad and collectors.time. _config.alerts switches single alerts off by name, and an unknown name fails the build. New alerts for the default collectors: WindowsCollectorTimeout, WindowsScrapeQueueTimeout, WindowsCPUSaturation (processor queue per core), WindowsMemoryPagingHigh, WindowsNetworkUtilizationHigh, WindowsNetworkInterfaceFlapping, WindowsServiceNotRunning (automatic services without triggers), WindowsServiceCrashLooping (processes per service in 15 minutes) and WindowsHostRebooted (info, fires once per reboot). WindowsNetworkErrors and WindowsNetworkDiscards compare the error and discard rates with the packet rate (networkErrorRatioThreshold and networkDiscardRatioThreshold, in percent) instead of an absolute packet rate. Threshold alerts set keep_firing_for from keepFiringFor (5m); the new infoSeverity and serviceSelector knobs follow the existing ones. New recordings: windows:cpu_queue_per_core:ratio, windows:net_received_packets:rate and windows:net_sent_packets:rate. Each new alert has a runbook, a README row and a unit test; tests/alerts.test.jsonnet checks the collector gating, the alert switches and the group layout. The runbook check renders the alerts of every collector. contrib/mixin/AGENTS.md documents the checklist for new alerts. Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
41 lines
1.5 KiB
Markdown
41 lines
1.5 KiB
Markdown
---
|
|
title: WindowsScrapeQueueTimeout
|
|
---
|
|
|
|
## Meaning
|
|
|
|
`windows_exporter_scrape_queue_timeout == 1` for ten minutes. Scrapes run one
|
|
at a time; a scrape whose timeout ends while it waits for an earlier scrape to
|
|
finish runs no collector and reports this metric as `1`. On such a scrape
|
|
`windows_exporter_collector_success` is `0` for every collector.
|
|
|
|
## Impact
|
|
|
|
The affected scrapes return no collector metrics, so the host has gaps in all
|
|
of its series and `WindowsCollectorFailed` can fire for every collector at
|
|
once. The exporter itself is reachable.
|
|
|
|
## Diagnosis
|
|
|
|
1. Confirm the queue timeout and find the scrape that holds the slot:
|
|
|
|
```promql
|
|
windows_exporter_scrape_queue_timeout{job="windows_exporter"} == 1
|
|
windows_exporter_collector_timeout{job="windows_exporter"} == 1
|
|
scrape_duration_seconds{job="windows_exporter"}
|
|
```
|
|
|
|
2. Check whether more than one Prometheus server or job scrapes the same
|
|
exporter, or whether a job scrapes it more often than its scrape duration
|
|
allows. Compare `scrape_duration_seconds` with the `scrape_interval`.
|
|
3. Identify the slow collector with `windows_exporter_collector_duration_seconds`
|
|
and follow `WindowsCollectorTimeout` for it.
|
|
|
|
## Mitigation
|
|
|
|
Scrape each exporter from one job, or give the jobs a `scrape_interval` longer
|
|
than a full scrape takes. Shorten the slow collector or raise the
|
|
`scrape_timeout` so a scrape finishes before the next one starts. Confirm
|
|
`windows_exporter_scrape_queue_timeout` stays `0` and collector success returns
|
|
to `1`.
|