Files
windows_exporter/docs/runbooks/windowscpusaturation.md
T
Jan-Otto Kröpke ae5d0cca2a mixin: group alerts by collector area and add tier 1 alerts
The alerts of the mixin now come in one rule group per collector area
(windows-exporter, windows-cpu, windows-memory, windows-disk,
windows-network, windows-service, windows-system, windows-active-directory,
windows-time). Every alert names the collectors it reads and is generated
only when _config.collectors enables them, the same switch the dashboard
uses; enableActiveDirectory and enableTime set collectors.ad and
collectors.time. _config.alerts switches single alerts off by name, and an
unknown name fails the build.

New alerts for the default collectors: WindowsCollectorTimeout,
WindowsScrapeQueueTimeout, WindowsCPUSaturation (processor queue per core),
WindowsMemoryPagingHigh, WindowsNetworkUtilizationHigh,
WindowsNetworkInterfaceFlapping, WindowsServiceNotRunning (automatic
services without triggers), WindowsServiceCrashLooping (processes per
service in 15 minutes) and WindowsHostRebooted (info, fires once per
reboot). WindowsNetworkErrors and WindowsNetworkDiscards compare the error
and discard rates with the packet rate (networkErrorRatioThreshold and
networkDiscardRatioThreshold, in percent) instead of an absolute packet
rate. Threshold alerts set keep_firing_for from keepFiringFor (5m); the new
infoSeverity and serviceSelector knobs follow the existing ones.

New recordings: windows:cpu_queue_per_core:ratio,
windows:net_received_packets:rate and windows:net_sent_packets:rate.

Each new alert has a runbook, a README row and a unit test;
tests/alerts.test.jsonnet checks the collector gating, the alert switches
and the group layout. The runbook check renders the alerts of every
collector. contrib/mixin/AGENTS.md documents the checklist for new alerts.

Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
2026-10-16 23:06:59 +00:00

40 lines
1.7 KiB
Markdown

---
title: WindowsCPUSaturation
---
## Meaning
The number of threads waiting for a processor, averaged over the rate window
and divided by the number of logical processors, exceeds the configured
threshold for 15 minutes. The default is two waiting threads per logical
processor. The metric is `windows:cpu_queue_per_core:ratio`, built from
`windows_system_processor_queue_length` and `windows:cpu_cores:count`.
## Impact
Runnable threads wait for CPU time, so applications respond slowly even when
their own CPU usage looks moderate. A long queue with CPU usage below 100%
points at a few busy cores, a single-threaded bottleneck, or processor
scheduling problems such as parked cores or hypervisor contention.
## Diagnosis
1. Inspect `windows:cpu_queue_per_core:ratio` next to `windows:cpu_usage:ratio`
for the host. Saturation with high usage is a capacity problem; saturation
with low usage points at scheduling.
2. Compare the per-core rates of `windows_cpu_time_seconds_total`. One or two
cores at 100% while others idle indicate a thread that cannot spread.
3. Check `windows_cpu_parking_status`, `windows_cpu_core_frequency_hertz` and
the power plan. On virtual machines check the hypervisor for CPU ready time
or steal.
4. Use Task Manager or Performance Monitor (`System\Processor Queue Length`,
`Process\Thread Count`) to find the processes with many runnable threads.
## Mitigation
Reduce or redistribute the workload, add logical processors, or fix the power
plan and parking settings. Give the virtual machine more CPU share or move it to
a less loaded host. Tune the threshold for hosts whose workload keeps a long
queue by design, such as build servers. Confirm the queue length per core and
application latency return to normal.