windows_exporter/AGENTS.md

Agent instructions

Follow CONTRIBUTING.md for all changes. These instructions apply throughout the repository; also read any AGENTS.md in the area you edit.

Project and source map

windows_exporter is a Go Prometheus exporter for Windows. Most collectors and native bindings require Windows; a successful cross-build does not validate their runtime behavior.

• cmd/windows_exporter: executable, flags, process settings, Windows service startup and shutdown.
• pkg/collector: collector registration, configuration, initialization, filtered views, scrape scheduling and resource ownership.
• internal/collector: built-in collectors. Each implements GetName, Build, Collect and Close from pkg/collector/types.go.
• internal/pdh: performance counter queries. internal/mi provides Management Infrastructure queries; internal/ole and internal/headers contain native Windows bindings.
• internal/config, internal/httphandler and internal/log: YAML/CLI configuration, HTTP scraping and logging.
• docs/collector.<name>.md: collector flags, metric names, types, labels and examples. docs/collector-template.md is the documentation template.
• contrib/mixin: Jsonnet recording rules, alerts and dashboard sources. dashboard contains the generated Grafana dashboards; docs/runbooks documents the alerts.
• installer, kubernetes and tools: MSI packaging, deployment examples, CI fixtures and end-to-end helpers. The CI package job validates installer changes with tools/test-installer.ps1, which installs the service for real. Never run it on a developer machine. Never upgrade from a 0.31.x package there: its taskkill /T follows stale parent process IDs and can kill the runner agent, which fails the job with "The hosted runner lost communication with the server" and no logs.
• Release builds give the executable and the MSI a numeric four-part version computed in the CI build job (output/FILE_VERSION): the fourth field is N for alpha.N/beta.N, 100+N for rc.N and 1000 for the release, so MDM tools that compare file versions update from one beta to the next and to the final release. Windows Installer itself ignores the fourth field.

New collectors need discussion with maintainers first. Prefer the configurable performancecounter collector where it can provide the requested metrics, as described in CONTRIBUTING.md.

Development and checks

Use the Go version specified in go.mod and the linter version pinned in Makefile. Do not change versions to work around a local tooling problem. Follow .golangci.yaml, .editorconfig and .gitattributes; keep LF line endings and the existing license headers. Use structured slog attributes and the existing Windows API wrappers (golang.org/x/sys/windows, NewLazySystemDLL). LazyProc.Call panics for a missing export: add every proc a binding calls without its own Find check to the package's lazyproc.Once list, and call the package's Load in the Build of each collector that uses it. Mirrors of native structs that MI or COM read by offset need unsafe.Sizeof/Offsetof checks, see internal/mi/layout.go.

Before committing, run the repository checks from its root:

make fmt
make lint
make test

If GNU Make is unavailable on Windows, use the equivalent commands from the current Makefile. Its current formatter and linter commands are:

go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.14.0 fmt
go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.14.0 run ./...
go test ./...

Release builds apply a go build -overlay from tools/go-overlay that removes the 32 MiB FIPS 140-3 entropy buffer Go links into every binary (golang/go#81956). It is pinned to the checksum of the patched standard library file, so a Go update that changes that file fails the build job until the overlay is refreshed or, once Go fixes the issue, removed. Plain go build and make build do not apply it.

For collector or concurrency changes, also run the affected packages natively on Windows with go test -race -count=1 -timeout=10m <packages>. Add regression tests for behavior changes and benchmarks for performance improvements. Check actual emitted metric values, types and labels through registry gathering; configuration or struct assertions alone do not prove publication is correct.

Read docs/ci-testing.md before changing fixtures or test skip rules. WINDOWS_EXPORTER_TEST_COLLECTORS lists collectors provisioned by CI; their failures must fail tests. Optional roles may be absent locally. Report skipped coverage and checks blocked by missing tools, permissions, services or network access in the PR. Do not claim that cross-compilation or a fixture test exercised a live Windows role.

Collector implementation rules

• Preserve metric names, label sets, types, units, defaults and configuration compatibility unless the change intentionally updates that contract. Update the corresponding collector documentation and affected queries together. Follow existing deprecation patterns when replacing published metrics.
• Determine metric types per counter, using pdh.Collector.MetricType where appropriate. Raw PDH values can be cumulative numerators, ticks or counts, even when the Windows counter name says average, percentage or seconds. Convert units explicitly; expose the base/operation count needed for ratios. The raw value of a PERF_AVERAGE_TIMER counter is in ticks of its time base: in new code, divide it by pdh.Collector.Frequency instead of a fixed factor. The disk and smbclient collectors still use the fixed 1e-7 (10 MHz) factor. Never publish an invalid sample as zero: that can look like a counter reset.
• Keep the typed PDH collectors and perfdata tags. internal/pdh passes instance names through unchanged, so a #N in a name always comes from Windows (IIS recycling, Ethernet#2). Some objects report one name several times per sample (Process, .NET CLR *, Exchange ADAccess processes, Hyper-V objects keyed by user-chosen VM or switch names). Rows without an Occurrence int field get only the first valid instance of a name, and pdh logs a warning once. Any object keyed by names users or applications choose can repeat them: declare the field there and choose the label value, usually with pdh.SuffixDuplicates. Those suffixes are not stable identities. Do not remove every name ending in _Total: legitimate process and database names can use that suffix. Use the shared aggregate-instance policy. pdh.Collector.Collect reuses the backing array of its destination slice and Row.Values; do not keep rows, sub-slices or pointers into it across scrapes.
• PDH collectors add counters by English path. On a Windows installation without en-US resources, the 009 name table holds localized names for v2 (manifest-based) counter sets such as SMB Server Shares or HTTP Service; internal/pdh/localized.go then maps the English names through the v2 registration API. Classic (v1) sets such as Processor are not affected.
• Keep Build, Collect and Close serialized for each collector instance. Filtered WithCollectors views share execution state and do not own shutdown; the original Collection owns collectors and MI resources. A timed-out call may still be running: prevent overlap, drain late metrics and do not free its resources while it runs. Honor the supplied scrape budget in blocking APIs. Build has no budget: bound its MI probe queries with mi.BuildQueryTimeout. A non-positive MI query timeout selects mi.DefaultQueryTimeout (5s), which is too short for slow providers.
• Collectors that reuse metrics or label pairs between scrapes keep an internal/metriccache or internal/labelcache field and list it in ScrapeCaches() (internal/scrapecache). The collection resets these caches before Build and after Close, and begins a scrape before and ends it after each Collect; Collect gets it with Scrape(). Do not call Begin, Commit or Reset in collectors. A direct Collect call, as in most tests, runs without the caches; use testutils.Collect or scrapecache.Begin to test the cached path.
• Release resources after partial initialization and on error paths. PDH constructors release what they opened and return a nil collector or query with an error, so close only what was returned without one; Close is nil-safe (see internal/pdh/AGENTS.md). Sub-collector cleanup closures must read the initialized field when called; method values captured before initialization can retain nil receivers. Stop background workers before releasing the resources they use, and recover panics inside the goroutine that can panic.
• Inspect every independent cause in joined errors. An expected no-data or unsupported-subsystem error must not hide an unrelated failure. Preserve valid metrics from successful parts of a scrape and the existing child-success metrics for collectors with sub-collectors.
• Match native function signatures, struct layouts and buffer-size units to the Windows API contract. Query required buffer sizes safely and retry when they grow. Distinguish bytes from UTF-16 characters, signed integers from bit patterns, and MI NULL values from populated unions. Respect borrowed/owned pointers, allocator-specific cleanup and COM thread initialization; use the native COM helpers rather than restoring the removed go-ole dependency. Run COM calls on an ole.Worker or another long-lived COM thread that Build starts and Close stops; initializing and uninitializing COM per scrape unloads in-process COM servers each time. A COM call cannot be interrupted: if Collect waits for one, wait without a deadline of its own (ole.Do with context.Background()), so that the collection's busy-skip handles a hung call instead of every later scrape queueing behind it.
• A collector's Close waits at most 3 seconds (closeTimeout) for its own workers, below the 5 seconds the collection gives all collectors to close. Document what a worker still running after that does with its resources.
• Keep CLI and YAML validation consistent. Reject unknown fields, unsupported sub-collector names and invalid values; preserve CLI precedence and repeated flag/list behavior. Changes to config shapes also need schema, flag parsing and documentation updates. A collector is registered once, in the registrations table of pkg/collector/map.go, with a Config field whose YAML key is the collector name and a docs/collector.<name>.md; TestRegistrations checks that they agree. The configuration file schema in internal/config only checks keys: the flags parse and validate the values. Register include/exclude expression flags with utils.RegexpFlag, which anchors the expression, stores the compiled *regexp.Regexp in the config and names the flag in parse errors; use utils.IsMatchAll/IsMatchNone to detect unfiltered defaults.
• For textfiles, keep HELP/type metadata consistent across files and preserve unrelated valid series when reporting duplicates or parse failures. Publish producer output by replacing a temporary file in the same directory. File mtime indicates accepted input; a producer's last-success timestamp indicates data freshness. Use emitted exporter/collector signals in alert examples.

These rules draw on the latest 150 PRs reviewed for this guide (#2451 through #2602, including open and closed proposals). Representative merged changes are scrape lifecycle #2541, filtered ownership #2587, PDH types and validity #2543, MI return values #2537, native COM #2551 and joined collection errors #2574. Check the current source and PR status before treating an open proposal as implemented behavior.

Live exporter validation

The development environment may already have an exporter running at http://localhost:9182. Check availability and windows_exporter_build_info before using it as a baseline; it may have a different revision, configuration or enabled collector set from your checkout.

(Invoke-WebRequest -Uri 'http://localhost:9182/metrics' -TimeoutSec 30).Content
(Invoke-WebRequest -Uri 'http://localhost:9182/metrics?collect[]=cpu' -TimeoutSec 30).Content

Inspect windows_exporter_collector_success, windows_exporter_collector_duration_seconds, windows_exporter_collector_timeout and windows_exporter_scrape_queue_timeout (the scrape timed out waiting for an earlier scrape, so no collector ran), plus the affected metric families. HTTP 200 alone does not mean every collector succeeded. PDH rate counters need a second sample after initialization.

To validate changed code, build a separate executable and run it on an unused port with the relevant collectors, then scrape it from another shell:

go build -o windows_exporter.dev.exe ./cmd/windows_exporter/
.\windows_exporter.dev.exe --web.listen-address=127.0.0.1:9183 --collectors.enabled=cpu

Adjust the collector list for the change. Leave the existing exporter instance running unless the user requests changes to it.

To check a collector's metric types against Windows, list each counter of the PDH object with its type and raw value. Windows PowerShell 5.1 can do this through .NET; RawValue and BaseValue are what pdh.CounterTypeRaw returns as first and second value:

$cat = [System.Diagnostics.PerformanceCounterCategory]'DirectoryServices'
$cat.GetCounters('NTDS') | ForEach-Object {
    $s = $_.NextSample()
    '{0} | {1} | raw={2} base={3}' -f $_.CounterName, $_.CounterType, $s.RawValue, $s.BaseValue
}

Every PDH-based collector except exchange, smtp and vmware has such a dump in the AGENTS.md of its directory, including roles that are rarely available on development machines (AD DS, AD CS, AD FS, DFSR, Hyper-V, IIS, SQL Server, NPS, RemoteFX sessions). Check a type or unit question there first.

Memory and leak investigations

Reports of growing memory are usually warm-up or bounded high-water marks, so measure before changing code:

• process_virtual_memory_bytes is the private (committed) bytes on Windows; process_resident_memory_bytes is the working set, which Windows trims, so it drops and recovers without any change in the process. Compare the daily minimum of the private bytes over several days. Go holds about go_memstats_sys_bytes - go_memstats_heap_released_bytes of it; the rest is native memory plus the executable's data and BSS sections, which include a 32 MiB buffer in builds without the release overlay (golang/go#81956).
• Go memory levels off within a few hours. The Go runtime keeps idle OS threads (Ms) parked instead of destroying them; threads it adds for blocking calls and MI callbacks stay, each with about six handles, so go_threads and process_open_fds rise in steps and then stay flat. A thread only exits when its goroutine ends while locked with runtime.LockOSThread, for example a closed ole.Worker. Investigate handles only if they grow while threads stay flat. go_sched_goroutines_not_in_go_goroutines rises on Windows on its own (golang/go#82136); it does not indicate a goroutine leak.
• --debug.enabled exposes /debug/pprof/. Diff two heap profiles taken hours apart for live Go memory and allocs profiles for allocation churn.
• Handle counts by object type need no admin rights: NtQuerySystemInformation(SystemExtendedHandleInformation) plus NtQueryObject(ObjectTypesInformation), for example from a PowerShell Add-Type script. handle.exe needs admin rights.

For native heap growth, use UMDH from the Debugging Tools for Windows in an elevated shell. It records an allocation stack for every heap allocation of processes started after the flag is set:

$env:_NT_SYMBOL_PATH = 'srv*C:\symbols*https://msdl.microsoft.com/download/symbols'
gflags /i windows_exporter.exe +ust
Restart-Service windows_exporter
# After warm-up, and again hours later:
umdh -p:<pid> -f:a.txt
umdh -p:<pid> -f:b.txt
umdh a.txt b.txt > diff.txt
# Afterwards:
gflags /i windows_exporter.exe -ust
Restart-Service windows_exporter

Snapshots from different processes cannot be compared. Stack recording cost about 24 MB for its database plus heap overhead on a development machine. Expected entries that are not leaks: single 1 MB data buffers of the perfos and perfts counter providers inside PdhCollectQueryData, and PDH counter path strings whose count grows until every instance has been seen. A leak adds allocations under the same stack in every snapshot.

UMDH does not see memory a library reserves with VirtualAlloc directly. A full dump shows it. cdb -pv attaches without changing the process and pauses it only while it writes the dump:

cdb -pv -p <pid> -c ".dump /ma C:\dumps\exporter.dmp; q"
cdb -z C:\dumps\exporter.dmp -c "!address -summary; !heap -s; q"

If the debugger cannot download a PDB, fetch it with cURL from https://msdl.microsoft.com/download/symbols/<name>.pdb/<id>/<name>.pdb into <store>\<name>.pdb\<id>\<name>.pdb and add <store> to the symbol path. <id> is the RSDS GUID of the module as 32 uppercase hex digits without dashes, followed by its age in hex.

Dashboards, rules and runbooks

Read contrib/mixin/README.md and dashboard/README.md before editing monitoring artifacts. Edit the Jsonnet/Grafonnet source under contrib/mixin/dashboards, including its tab modules and shared builders, and regenerate dashboard/windows-exporter-*.json: the full dashboard windows-exporter-dashboard.json and one dashboard per tab. Do not hand-edit generated JSON. Tab dashboards keep only the variables their panels reference, so reference variables as $name or ${name}. Gate every tab, row, panel and variable on the collectors it queries with on('<collector>') (names from _config.collectors), and run jsonnet -J vendor tests/dashboard.test.jsonnet to check the layout for several collector selections.

From contrib/mixin, install the tools pinned in mise.toml with mise install, then run mise run fmt, mise run generate, mise run check-dashboards and mise run test. Keep generated artifacts in sync. The tests cover Jsonnet format, dashboard parity, promtool rules/tests, offline pint lint, dashboard lint and runbook coverage.

The tasks are POSIX shell scripts (set -eu, trap, mktemp, $(...)) that also call rm, cp, diff, git and, for check-runbooks, python3. On Windows, mise runs inline tasks with cmd /c by default, so run them in WSL, or from Git Bash with MISE_WINDOWS_DEFAULT_INLINE_SHELL_ARGS="bash -c" and Git's bash (not C:\Windows\System32\bash.exe, which starts WSL) first on PATH.

Without mise, run the Jsonnet tools with go run at the versions pinned in mise.toml, for example go run github.com/google/go-jsonnet/cmd/jsonnet@v0.22.0 (also jsonnetfmt and jsonnet-lint) and go run github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@v0.6.0 install. That covers formatting, generation and the Jsonnet tests; promtool, pint and dashboard-linter still need their release binaries, so report those checks as skipped. A Windows checkout with core.autocrlf has CRLF sources, so compare generated output with diff --strip-trailing-cr.

go test ./internal/metriccontract/ checks that every windows_ metric name in the readme files, docs/collector.*.md, the runbooks, the mixin and the dashboards is created by a collector; it reads the Go sources, so it also runs on Linux. When documentation names a renamed or removed metric on purpose, add it to historicalReferences in that test with the reason.

Preserve target/device labels, apply counter rates before aggregation, and handle zero denominators. Keep optional-role alerts configurable and provide a runbook for each alert. Check required collector availability and the Grafana version documented by the dashboard.

Pull requests

Branch from current upstream main, keep commits focused and sign off each commit with git commit -s for the DCO. Preserve unrelated local changes; use a separate worktree when needed. Follow .github/PULL_REQUEST_TEMPLATE.md and use the subsystem: description title format required by CONTRIBUTING.md and .github/workflows/pr-check.yaml. Explain the problem, resulting behavior and validation, including any limits.

Pull request labels

Before merging, apply at least one label from .github/release.yml:

• 💥 breaking-change: breaking changes
• ✨ enhancement: user-visible functionality
• 🐞 bug: fixes
• 🛠️ dependencies: dependency updates
• 📖 docs: documentation
• chore: internal performance work, tests, CI, or maintenance

Add 💥 breaking-change alongside any other applicable label. Labeling is an AI-agent-only operation; do not tell external contributors to apply labels.

Keep this guide accurate when an authorized change reveals a useful repository convention or pitfall. Put detailed component guidance in a scoped AGENTS.md when needed, and keep CLAUDE.md as a pointer to the shared instructions.