The alerts of the mixin now come in one rule group per collector area
(windows-exporter, windows-cpu, windows-memory, windows-disk,
windows-network, windows-service, windows-system, windows-active-directory,
windows-time). Every alert names the collectors it reads and is generated
only when _config.collectors enables them, the same switch the dashboard
uses; enableActiveDirectory and enableTime set collectors.ad and
collectors.time. _config.alerts switches single alerts off by name, and an
unknown name fails the build.
New alerts for the default collectors: WindowsCollectorTimeout,
WindowsScrapeQueueTimeout, WindowsCPUSaturation (processor queue per core),
WindowsMemoryPagingHigh, WindowsNetworkUtilizationHigh,
WindowsNetworkInterfaceFlapping, WindowsServiceNotRunning (automatic
services without triggers), WindowsServiceCrashLooping (processes per
service in 15 minutes) and WindowsHostRebooted (info, fires once per
reboot). WindowsNetworkErrors and WindowsNetworkDiscards compare the error
and discard rates with the packet rate (networkErrorRatioThreshold and
networkDiscardRatioThreshold, in percent) instead of an absolute packet
rate. Threshold alerts set keep_firing_for from keepFiringFor (5m); the new
infoSeverity and serviceSelector knobs follow the existing ones.
New recordings: windows:cpu_queue_per_core:ratio,
windows:net_received_packets:rate and windows:net_sent_packets:rate.
Each new alert has a runbook, a README row and a unit test;
tests/alerts.test.jsonnet checks the collector gating, the alert switches
and the group layout. The runbook check renders the alerts of every
collector. contrib/mixin/AGENTS.md documents the checklist for new alerts.
Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
Jan-Otto Kröpke