Jan-Otto Kröpke
d6afd48819
WindowsHostRebooted compares against the last boot time of the previous hour instead of the sample of ten minutes ago, so a scrape gap during the reboot no longer suppresses the alert. WindowsServiceNotRunning joins with ignoring() so instances with the same name in different clusters do not match each other. Replacing the removed networkErrorRateThreshold or networkDiscardRateThreshold knobs now fails with a message naming the ratio thresholds. The network error and discard runbooks describe the packet ratio, the service runbooks explain PID reuse, shared host processes and selector exclusions, and the tests cover alert recovery, a reboot after a scrape gap and a service name repeated across clusters. Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
1.9 KiB
title
| title |
|---|
| WindowsNetworkDiscards |
Meaning
Discarded inbound and outbound packets make up more than the configured share
of all packets on an interface for 15 minutes. The default
networkDiscardRatioThreshold is 1 percent. The ratio divides the discard rates
(windows:net_received_discarded:rate plus windows:net_outbound_discarded:rate)
by the packet rates (windows:net_received_packets:rate plus
windows:net_sent_packets:rate). Discards count packets dropped without a packet
error, such as buffer exhaustion. Interfaces that carry no packets produce no
sample and cannot alert. The alert keeps firing for keepFiringFor after the
ratio drops below the threshold.
Impact
Dropped packets can cause retransmissions, reduced throughput, and application timeouts. Some drops may be expected under a deliberate traffic policy.
Diagnosis
- Inspect
windows:net_received_discarded:rateandwindows:net_outbound_discarded:rateto identify the direction and affectednic, and compare them with the packet rate recordings to see whether a low packet rate inflates the ratio. - Compare traffic, directional utilization ratios, and
windows_net_output_queue_length_packetswith the normal baseline. - Inspect
Get-NetAdapterStatistics, switch-port drops, adapter buffers, virtual switch counters, and network QoS policies. - Correlate drops with CPU pressure, traffic bursts, hypervisor contention, and application failures. An unknown or zero bandwidth estimate produces no utilization sample and should not be interpreted as zero traffic.
Mitigation
Relieve congestion, distribute traffic, or add bandwidth. Correct buffer, driver,
or virtual switch problems when evidence identifies them. Review intentional
drop policies with the network owner before raising
networkDiscardRatioThreshold. Confirm discard rates and application
retransmissions return to the expected baseline.