container: collect Hyper-V isolated containers from CRI stats (#2568)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
This commit is contained in:
Jan-Otto Kröpke
2026-10-09 09:15:51 +02:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 200f248eba
commit fedca642a1
7 changed files with 649 additions and 36 deletions
+28 -2
View File
@@ -15,6 +15,7 @@ Enabled by default? | No
Comma-separated list of collectors to use. Defaults to all, if not specified. Available collectors: `hcs`, `hostprocess`.
- `hcs` collects containers managed by the Host Compute Service, for example Docker or process-isolated containerd containers.
Kubernetes containers with Hyper-V isolation are collected from the CRI endpoint, see [Hyper-V isolated containers](#hyper-v-isolated-containers).
- `hostprocess` collects Kubernetes HostProcess containers, which run in Win32 job objects instead of HCS.
### `--collector.container.cri-endpoint`
@@ -22,7 +23,8 @@ Comma-separated list of collectors to use. Defaults to all, if not specified. Av
Kubernetes Container Runtime Interface (CRI) endpoint. Defaults to `npipe:////./pipe/containerd-containerd`, the endpoint of containerd.
The collector reads the running containers and pod sandboxes from this endpoint to add the `namespace`, `pod` and `container` labels, to skip pause containers and to find HostProcess containers.
If the endpoint is not available, for example on hosts without Kubernetes, the HCS containers are exported without Kubernetes labels and HostProcess containers aren't collected.
It also reads the container stats for the writable layer usage and the metrics of Hyper-V isolated containers.
If the endpoint is not available, for example on hosts without Kubernetes, the HCS containers are exported without Kubernetes labels and HostProcess and Hyper-V isolated containers aren't collected.
## Metrics
@@ -49,11 +51,35 @@ If the endpoint is not available, for example on hosts without Kubernetes, the H
| `windows_container_storage_read_size_bytes_total` | Read Size Bytes | counter | `container_id`,`namespace`,`pod`,`container` |
| `windows_container_storage_write_count_normalized_total` | Write Count Normalized | counter | `container_id`,`namespace`,`pod`,`container` |
| `windows_container_storage_write_size_bytes_total` | Write Size Bytes | counter | `container_id`,`namespace`,`pod`,`container` |
| `windows_container_storage_writable_layer_usage_bytes` | Used bytes of the writable layer | gauge | `container_id`,`namespace`,`pod`,`container` |
`windows_container_start_time_seconds` is the start time reported by HCS. HostProcess containers have no start time, they report their creation time from the CRI endpoint instead.
`windows_container_start_time_seconds` is the start time reported by HCS. HostProcess and Hyper-V isolated containers have no start time in HCS on the host, they report their creation time from the CRI endpoint instead.
`windows_container_memory_page_faults_total` is only available for HostProcess containers, because HCS doesn't report page faults. The job object counts page faults in 32 bits, so the counter wraps around after 2^32 page faults.
`windows_container_storage_writable_layer_usage_bytes` is read from the CRI endpoint and only exported for Kubernetes containers.
containerd measures it periodically, so it may lag behind and is missing until the first measurement.
`windows_container_count` only counts HCS containers. HostProcess and Hyper-V isolated containers aren't counted.
### Hyper-V isolated containers
Kubernetes containers with Hyper-V isolation run inside a utility VM. HCS on the host only sees the utility VM, not the containers inside it.
The collector reads their metrics from the container stats of the CRI endpoint instead, if the `hcs` collector is enabled.
containerd fills them from the HCS statistics that the shim queries inside the utility VM, but the CRI API only carries a subset of them:
| Name | Source |
|------------------------------------------------------------|----------------------------------------|
| `windows_container_available` | always `1`, with `hostprocess="false"` |
| `windows_container_cpu_usage_seconds_total` | `cpu.usage_core_nano_seconds` |
| `windows_container_memory_usage_private_working_set_bytes` | `memory.working_set_bytes` |
| `windows_container_memory_usage_commit_bytes` | `memory.usage_bytes` |
| `windows_container_start_time_seconds` | creation time of the container |
| `windows_container_storage_writable_layer_usage_bytes` | `writable_layer.used_bytes` |
The user and kernel mode CPU time, the commit peak, page faults, the process count, storage I/O and network metrics aren't available for Hyper-V isolated containers.
The collector identifies them as running Kubernetes containers that are neither HCS containers nor HostProcess containers.
### Example metric
_windows_container_network_receive_bytes_total{container_id="docker://1bd30e8b8ac28cbd76a9b697b4d7bb9d760267b0733d1bc55c60024e98d1e43e",interface="822179E7-002C-4280-ABBA-28BCFE401826"} 9.3305343e+07_