mirror of
https://github.com/prometheus-community/windows_exporter.git
synced 2026-10-17 10:09:05 +02:00
Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
5.6 KiB
5.6 KiB
scheduled_task collector
Scrape time is spent in the Task Scheduler service. The exporter's own CPU
time is small, but the Schedule service (svchost) uses about as much CPU time
as the scrape takes. Measure that with the raw PercentProcessorTime of
Win32_PerfRawData_PerfProc_Process for the service's PID (100 ns units),
which an unprivileged user can read.
IRegisteredTaskNameandPathare answered locally from the enumerated collection.State,Enabled,LastTaskResultandNumberOfMissedRunseach make a separate RPC to the Schedule service (roughly 0.1–0.3 ms per call).ITaskFolder::GetTasksandGetFoldersadd one RPC per folder each.IRegisteredTaskCollection::get_Itemis local, so enumerating with_NewEnumand batchedIEnumVARIANT::Nextsaves nothing.- Keep the serial walk. Reading folders with concurrent workers, each with its own connection, halved the scrape time but cost the Schedule service about 40% more CPU time in total with four workers. That is a bad trade on small VMs (#2643). Re-opening folders by path instead of walking the folder objects also adds a round trip per folder.
- Read only the properties that are published, apply the include/exclude filter to the path before any service read, and keep property reads conditional on whether the metric that uses them is emitted.
folderFilterskips subfolders whose tasks the include/exclude expressions reject whatever their names are, by analyzing the parsed expressions. It must stay conservative: when in doubt, walk the folder. A Windows 11 desktop has about 150 folders for about 200 tasks, and listing the tasks and subfolders of each costs ~0.15 ms per call, soexclude=/Microsoft/.+saves most of a scrape. The default exclude is/Microsoft/.+for that reason; keepConfigDefaultsand the flag default in sync throughdefaultTaskExclude.taskReadercarries the task and folder filters and which properties to read.NumberOfMissedRunsis only read and published withmissed-runs(off by default), which saves its two RPCs per task.- The fake native task in
scheduled_task_native_test.gocounts reads per vtable slot; extend those assertions when adding a property read. - To find slow calls, time each call type over a full enumeration on a live
system. Run it as
SYSTEMas well as a user: unprivileged accounts see fewer tasks. - Each task publishes 19 const metrics, and building them is most of the
allocations of a scrape.
collectMetricsreuses the metrics of the previous scrape throughinternal/metriccachewhile a task's path, state, last result and missed runs are unchanged. Add new inputs of a task's metrics totaskValues, or the cache returns stale series.
Measured costs and rejected approaches
Measured in October 2026 on a Windows 11 24H2 desktop (taskschd.dll 10.0.26100.5074), unprivileged: 207 tasks in 150 folders (134 leaf folders, 36 of them empty), about 150 ms per scrape without filters.
- Per call:
GetTasks~170 µs andGetFolders~145 µs per folder;StateandLastTaskResult~130 µs per task;NumberOfMissedRuns~310 µs per task (~560 µs for disabled tasks).Pathand folderPathtake 1–2 µs (local).NumberOfMissedRunsalone is about 28% of a scrape. NumberOfMissedRunsis computed with two RPCs:SchRpcGetLastRunInfo(opnum 16), thenSchRpcScheduledRuntimes(opnum 15) from the start of the last run to the current local time with an unlimitedcRequested. The service computes every scheduled run in that window from the task definition and returns all of them, so tasks with many missed runs (13,146 seen) send large responses each scrape. Tasks without time triggers returnSCHED_S_TASK_NOT_SCHEDULED.LastTaskResultalso callsSchRpcGetLastRunInfo, so reading both costs one duplicate round trip.SchRpcGetNumberOfMissedRuns(opnum 18) always returns 0 and is not what COM uses.- Missed runs are rare but real: 6 of 207 tasks were non-zero, 2 of them disabled. Skipping the read for disabled tasks would change published values.
- A persistent COM connection (dedicated thread, service kept connected) saves
~1.7 ms per scrape.
CoCreateInstanceandConnectare cheap; the service's per-call time dominates. - Direct MS-TSCH RPC over
ncalrpc(hand-marshalled NDR,I_RpcSendReceive) is only 5–8 µs faster per call. It would only save the duplicateSchRpcGetLastRunInfo(~18 ms per scrape, ~12%) plus ~9 ms of client CPU, at the cost of a hand-written NDR client. Not worth it. Note thatSchRpcGetTaskInfoonly returns the state withSCH_FLAG_STATE(0x10000000). - One
SchRpcEnumInstancesfor all running tasks took ~21 ms, slower than the per-task state reads. - There is no flat or recursive task enumeration:
GetTasks/GetFoldersandSchRpcEnumTasks/SchRpcEnumFolderslist one folder each, with only the hidden-task flag. The registryTaskCacheis undocumented and not readable without admin rights. MSFT_ScheduledTask(CIM,root/Microsoft/Windows/TaskScheduler) returns a flat list with path and state, but is about 3× slower: ~440 ms throughinternal/miin-process for the same 207 tasks, whatever properties are selected, plus ~120 ms for last result and missed runs. The work moves intoWmiPrvSE(up to ~30% of one core while querying).missed_runsis opt-in throughmissed-runs. Not taken yet: a cache duration that serves the last task list.- Profiling the exporter with pprof on Windows attributes the time blocked in
these COM calls to
runtime.cgocall; that is waiting, not exporter CPU. The Schedule service's own CPU needs an elevated reader of its process times.