Jan-Otto Kröpke
d819362150
Every optional collector of the mixin now has alerts, generated only with that collector enabled in _config.collectors: clock source, disk drive status, page file usage, license status, scheduled task results, pending updates, Hyper-V WMI and replication health, failover cluster nodes, resources, groups and shared volumes, Storage Spaces pool and virtual disk health, DHCP scope exhaustion and DNS zone transfer failures. The alerts of the pagefile, diskdrive and license collectors join the memory, disk and system groups; the other collectors get a group of their own. New knobs: taskSelector, sharedVolumeSelector, pagefileUsageThreshold, dhcpScopeFreeWarningThreshold, dhcpScopeFreeCriticalThreshold and updatePendingDuration. The test configuration enables every optional collector and tests/optional.test.yaml covers the new alerts. Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
1.9 KiB
title
| title |
|---|
| WindowsDiskDriveNotOK |
Meaning
The physical drive device_id reports an operational status other than OK
for five minutes. The alert carries the status, such as Pred Fail, Degraded
or Error, in its status label. It needs the diskdrive collector, which
reads the status that the drive and its driver report through WMI. The
Unknown status does not raise the alert: some USB and virtual drives report
it permanently. Transient statuses such as Starting or Stopping raise it
only when they last longer than five minutes.
Impact
Pred Fail is the drive's own prediction that it is about to fail. Any other
status means the drive is not fully working. Data on a single drive is at risk;
a drive in a mirror or parity set leaves the set without redundancy until it is
replaced.
Diagnosis
-
Inspect
windows_diskdrive_statusfor the host and check whether the volumes on the drive still report normal latency withwindows:logical_disk_read_latency:secondsandwindows:logical_disk_write_latency:seconds. -
On the host, read the drive and its reliability counters:
Get-CimInstance Win32_DiskDrive | Select-Object DeviceID, Model, Status Get-PhysicalDisk | Select-Object FriendlyName, HealthStatus, OperationalStatus Get-PhysicalDisk | Get-StorageReliabilityCounter | Select-Object DeviceId, ReadErrorsTotal, WriteErrorsTotal, Temperature, Wear -
Check the System event log for disk, storahci, stornvme or RAID controller events. On a virtual machine, check the hypervisor and the datastore of the virtual disk.
Mitigation
Back up the data on the drive first. Replace a drive that predicts its failure
or reports errors through the hardware or virtualization team. For a drive in
a storage pool or RAID set, follow the vendor's replacement and rebuild
procedure and confirm the set is redundant again. The alert resolves when the
status returns to OK.