[AUM 7.1] Build Sheet: The Operations Scaffolding

The queries, the alert, the export and the pre-event pair that every Update Manager estate ends up building. Reproducible from the text, with the timing rules that cancel a patch cycle if you ignore them.


By the end of this you have four Resource Graph queries that answer the questions people actually ask, one alert rule that fires on a failed run and has been proven to fire, an export that survives the thirty day cliff, and a pre and post event pair that wakes APP01 before its window and confirms it back down afterwards. This sheet is verified against documentation rather than run in a lab. I say that once, here, and then get on with it.


Before step one

You need the rings already standing: at least one maintenance configuration with scope InGuestPatch that has completed a real run, periodic assessment enabled, every target machine Connected. You need Reader at the scope you query, Contributor on the resource group holding the alert rule, Automation account and storage account, the Az.ResourceGraph module or the Azure CLI resource-graph extension, and the Microsoft.EventGrid provider registered before the second half. And remember that every unwaivered Arc machine that is assessed or associated with a schedule meters.

Name things before you create them, because half of these objects get discovered by wildcard six months from now by somebody who was not in the room.

ObjectPatternExample
Saved Resource Graph queryaum-scope-questionaum-estate-assessment-state
Alert ruleaum-alert-condition-scopeaum-alert-failed-runs-csj
Action groupag-aum-purposeag-aum-patching
Automation accountaa-org-purposeaa-csj-patching
Hybrid worker grouphwg-org-sitehwg-csj-onprem
RunbookPre-Verb or Post-VerbPre-WakeScheduledMachines
Automation webhookwh-phase-purpose-ringwh-pre-wake-ring2
Event subscriptionevt-phase-ring-purposeevt-pre-ring2-wake
Evidence storage accountstorgpatchevidencestcsjpatchevidence
Evidence blob pathpatch-history/table/yyyy/MM/ddpatch-history/installation/2026/08/12
Wake-state tagPatchWakeStateWokenByPreEvent

Tag values are case sensitive and dynamic scopes filter on them, so WokenByPreEvent is not the same value as wokenbypreevent.


1. The four queries

Do this in the portal first, because the portal is where you will edit them later. Open Azure Resource Graph Explorer, set the scope selector to the subscriptions holding your machines, paste the query and select Run query. Keep each one with Save as. Download as CSV sits on the same toolbar and is the fastest ad hoc export, capped at 55,000 records.

Query one, the estate’s assessment state. Microsoft’s own published sample, and I would not improve on it.

patchassessmentresources
| where type !has "softwarepatches"
| extend prop = parse_json(properties)
| extend lastTime = properties.lastModifiedDateTime
| extend criticalCount = prop.availablePatchCountByClassification.critical,
         securityCount = prop.availablePatchCountByClassification.security,
         updatesCount  = prop.availablePatchCountByClassification.updates,
         OS = prop.osType
| project lastTime, id, OS, criticalCount, securityCount, updatesCount
| order by lastTime asc

Expected: one row per assessed machine, oldest first, so the machines worth worrying about are at the top. Failure mode: an empty result on Arc machines is usually not a query fault, it is the subscription not being registered to the Microsoft.Compute resource provider, which silently produces no Arc assessment data.

Query two, failed runs. The alert is built on this, so get it right where you can see the output.

patchinstallationresources
| where type !has "softwarepatches"
| extend machineName = tostring(split(id, "/", 8)),
         rgName = tostring(split(id, "/", 4))
| extend prop = parse_json(properties)
| extend lTime = todatetime(prop.lastModifiedDateTime),
         status = tostring(prop.status),
         failedPatchCount = toint(prop.failedPatchCount),
         windowExceeded = tostring(prop.maintenanceWindowExceeded),
         errorDetails = tostring(prop.errorDetails)
| where lTime > ago(7d)
| where status in~ ("Failed", "CompletedWithWarnings")
     or failedPatchCount > 0
     or windowExceeded =~ "true"
| project lTime, machineName, rgName, status, failedPatchCount, windowExceeded, errorDetails
| order by lTime desc

Expected: zero rows on a healthy estate. It catches three bad outcomes: Failed, CompletedWithWarnings, which is what a pending reboot under a never-reboot policy produces, and maintenanceWindowExceeded true, which is the silent skip. An alert on status alone misses the third, and the third is the likeliest.

Query three, machines waiting on a reboot.

patchinstallationresources
| where type !has "softwarepatches"
| extend machineName = tostring(split(id, "/", 8))
| extend prop = parse_json(properties)
| extend lTime = todatetime(prop.lastModifiedDateTime),
         rebootStatus = tostring(prop.rebootStatus),
         installedPatchCount = toint(prop.installedPatchCount)
| where lTime > ago(30d) and rebootStatus in~ ("Required", "Failed")
| project lTime, machineName, rebootStatus, installedPatchCount
| order by lTime desc

Expected: rows only where the last run left a restart outstanding. rebootStatus is what the run recorded, not a live reading of the operating system, so a machine restarted by hand still appears until its next run overwrites it. A to-do list, not current state.

Query four, machines that have gone quiet. The trap in this one is worth more than the query.

patchassessmentresources
| where type !has "softwarepatches"
| extend machineName = tostring(split(id, "/", 8))
| extend lastAssessed = todatetime(properties.lastModifiedDateTime)
| where lastAssessed < ago(2d)
| project machineName, lastAssessed, id
| order by lastAssessed asc

Expected: empty, because periodic assessment runs every twenty four hours. Now the trap. Assessment records are retained seven days, so a machine silent for eight days does not appear as stale. It leaves the table entirely and this query reports it fine by omission. The complete answer reconciles the table against your machine inventory.

resources
| where type in~ ("microsoft.hybridcompute/machines", "microsoft.compute/virtualmachines")
| project machineId = tolower(id), machineName = name, machineType = type
| join kind=leftouter (
    patchassessmentresources
    | where type !has "softwarepatches"
    | extend machineId = tolower(tostring(split(id, "/patchAssessmentResults/")[0]))
    | extend lastAssessed = todatetime(properties.lastModifiedDateTime)
    | project machineId, lastAssessed
  ) on machineId
| project machineName, machineType, lastAssessed
| where isnull(lastAssessed) or lastAssessed < ago(2d)
| order by lastAssessed asc

That is a pattern rather than a query I have watched return rows. The identifier split follows Microsoft’s documented log structure, where an assessment record’s ID is the resource path followed by patchAssessmentResults and latest. Validate it in your own tenant, and if the join returns empty, project both machineId columns and compare.

The command line equivalent, for a script rather than a portal tab. Keep the query text in files beside the script so the portal copy and the automated copy stay identical.

az extension add --name resource-graph --only-show-errors

for q in assessment-state failed-runs pending-reboot gone-quiet; do
  az graph query -q "@queries/$q.kql" \
     --subscriptions "$SUBSCRIPTION_ID" --first 1000 \
     --output json > "out/$q.json"
done

Expected: four JSON files, each with a data array. Failure mode: a result stopping at exactly 1000 rows is truncated, because Resource Graph returns at most 1000 records per request and the rest are paged with a skip token, which the export below does properly.


2. One alert that fires on a failed run

Build exactly one alert to start with. An estate with six alerts on day one has six alerts nobody trusts by day thirty.

  1. In the Azure portal, open Azure Update Manager. Under Monitoring, select New alerts rule (Preview).
  2. Choose the Subscription that defines the rule’s scope, then the Resource Group and Location where the alert rule resource is created. That is the rule’s home, not the machines it watches.
  3. In the query dropdown select Custom query, then Skip to custom alert rules, to reach the full creation flow.
  4. Select Preview or edit query in Logs, paste query two, and select Run. Confirm the result set, then Continue Editing Alert.
  5. On Scope and filters, confirm the scope covers the subscriptions holding your machines and no more.
  6. Set measurement, threshold and evaluation frequency from the table below.
  7. On Actions, attach an action group named ag-aum-patching, with at least one email or push target.
  8. On the Details tab set Alert rule name to aum-alert-failed-runs-csj. That name is how you find Update Manager alerts among everything else in Azure Monitor.
  9. Set Identity to a managed identity, note its object ID, then Review + create.
  10. Assign that identity Reader at the scope containing the machines. Do this before you rely on the rule.
SettingValueWhy
Search queryQuery two, failed runsThree bad outcomes in one signal
MeasureTable rowsCounting bad runs, not aggregating a number
Aggregation typeCountSame reason
OperatorGreater thanAny row is a real event here
Threshold value0One failed run is worth a message. Raise it once you know your normal
Frequency of evaluation1 hourPatch windows are hours long. Faster evaluation buys noise
Severity2 (Warning)Not an outage. Keep 0 and 1 for things that page somebody
Automatically resolve alertsEnabledClears when the next run succeeds
IdentityManaged identity with Reader on the target scopeDocumented prerequisite. Without it the rule evaluates against nothing

Two constraints to design around rather than discover. The query behind an alert returns at most 1000 rows, and returns only what the managed identity can see, so a rule that looks correctly scoped can be silently narrowed by a missing role assignment. Neither produces an error. Both produce a quiet alert on an estate you thought was covered.


3. The export that outlives the thirty day cliff

Microsoft’s guidance is to build a process to export the data to a store you control. The cheapest version that works is a scheduled runbook in the Automation account you are about to build anyway, running the queries with paging and writing to blob storage under a dated path. Run it daily: assessment data survives only seven days.

Connect-AzAccount -Identity

$query = Get-Content -Raw -Path ".\queries\failed-runs.kql"
$all   = [System.Collections.ArrayList]@()
$skipToken = $null

do {
    $res = Search-AzGraph -Query $query -First 1000 -SkipToken $skipToken -Subscription $SubscriptionIds
    $skipToken = $res.SkipToken
    if ($res.Data) { $all.AddRange($res.Data) }
} while ($skipToken -ne $null -and $skipToken.Length -ne 0)

$stamp = (Get-Date).ToUniversalTime().ToString("yyyy/MM/dd")
$file  = "$env:TEMP\installation.json"
$all | ConvertTo-Json -Depth 10 | Out-File $file -Encoding utf8

Set-AzStorageBlobContent -File $file -Container "patch-history" -Blob "installation/$stamp/installation.json" -Context $ctx -Force

Expected: one blob per query per day, and a job history showing Completed. Failure modes in the order you meet them. A loop that returns 1000 records and stops is not carrying the skip token between iterations. A query returning nothing means the runbook identity lacks Reader on the subscriptions you passed. Put a delete lock or immutability on the container, because an export you can quietly overwrite is a copy, not evidence.

FileOriginDestinationPurpose on arrival
Four .kql query filesYour workstation, in source controlPackaged with the runbookPortal copy and automated copy stay identical
Export runbook .ps1Your workstationImported into aa-csj-patching, publishedRuns the paged export daily
Pre and post runbook .ps1Adapted from Microsoft’s samplesImported into aa-csj-patching, publishedHandle the maintenance events
Daily JSON exportsProduced in Azure by the runbookstcsjpatchevidence, patch-historyEvidence beyond 30 days
Webhook URLGenerated by Azure at creationThe event subscription, then a secret storeCannot be retrieved again

4. The pre and post event pair

These timings decide whether a patch cycle happens, so they get their own table. Read it before touching a schedule that carries an event.

RuleRelative to window startIf you miss it
Create or edit a schedule that has a pre-eventAt least 40 minutes beforeThe upcoming run auto-cancels. No patching that cycle
Pre-event is triggeredAt least 30 minutes beforeNot yours to control. This is when your handler starts
All pre-events completeWithin 20 minutes of triggeringThe run proceeds anyway unless you cancel it
Cancellation call landsAt least 10 minutes beforeThe run proceeds and installs updates
Machines powered onAt least 15 minutes beforeThe machine is skipped and the ring still reports success
Post-event firesWhen installation completesMay fall outside the window. Budget wall clock time for it

The sentence to internalise, in Microsoft’s words: you must initiate cancellation as part of the pre-event, Update Manager will not automatically cancel the schedule, and if you fail to cancel, the run proceeds. A pre-event that errors out stops nothing. If your handler is meant to be a gate, you write the gate.

Now the part the published samples do not cover. Microsoft’s sample start and stop runbooks filter the run’s machine list for Compute virtual machine resource IDs and call the Azure power cmdlets. That works for Azure virtual machines. Azure has no power control over an Arc-enabled server, so waking APP01 cannot be done from an Azure sandbox at all. The wake happens inside your network, and the documented mechanism is a Hybrid Runbook Worker, supported on Arc-enabled servers and selected with the Run on setting the webhook carries.

  1. Create the Automation account aa-csj-patching with a system-assigned managed identity. Grant it Virtual Machine Contributor, or a custom role carrying only the start, deallocate, restart and power off actions, on the resource group holding any Azure virtual machines in scope, plus Reader where it runs queries.
  2. Deploy an extension-based Hybrid Runbook Worker onto an always-on machine in the office, in a group named hwg-csj-onprem. Not a machine you are about to patch.
  3. Create and publish three PowerShell 7.4 runbooks: Pre-WakeScheduledMachines, Post-RestoreMachineState and Pre-CancelRun. The third exists so cancellation is something you own rather than write in a hurry later.
  4. Add a webhook to each runbook, setting Run on to hwg-csj-onprem for the two that touch on-premises machines. Copy each URL immediately. Azure will not show it again.
  5. Open the ring’s maintenance configuration, under Settings select Events, then Event Subscription. Keep the schema as Event Grid Schema, set Filter to Event Types to Pre Maintenance Event or Post Maintenance Event, choose the Web Hook endpoint and configure it with the URL you copied.

Every runbook starts by proving the event is one it should act on, because a webhook endpoint receives other subscription events if anything else is ever pointed at it.

param (
    [Parameter(Mandatory=$false)]
    [object] $WebhookData
)

Connect-AzAccount -Identity

$payload   = ConvertFrom-Json -InputObject $WebhookData.RequestBody
$eventType = $payload[0].eventType

if ($eventType -ne "Microsoft.Maintenance.PreMaintenanceEvent") {
    Write-Output "Not a pre-maintenance event. Exiting without action."
    return
}

$maintenanceRunId = $payload[0].data.CorrelationId
$subscriptionIds  = $payload[0].data.ResourceSubscriptionIds

$argQuery = @"
maintenanceresources
| where type =~ 'microsoft.maintenance/applyupdates'
| where properties.correlationId =~ '$($maintenanceRunId)'
| project id, resourceId = tostring(properties.resourceId)
| order by id asc
"@

$machines = Search-AzGraph -Query $argQuery -First 1000 -Subscription $subscriptionIds
Write-Output "Machines in this run: $($machines.Count)"

Notice what is not printed in that script. The correlation ID is generated per run and arrives in the payload, so it is read and used as a variable. Same discipline for the other two names Azure generates here: the webhook URL, shown once, and the Event Grid system topic created with your first event subscription.

az eventgrid system-topic list --resource-group "$RESOURCE_GROUP" \
   --query "[].{name:name, source:source, topicType:topicType}" --output table

az eventgrid system-topic event-subscription list \
   --resource-group "$RESOURCE_GROUP" \
   --system-topic-name "$SYSTEM_TOPIC_NAME" --output table

Expected: one system topic per maintenance configuration carrying events, with your named subscriptions under it. Failure mode: an empty list usually means the event subscription landed in a different resource group from the maintenance configuration.

Verify before the irreversible step, not after it. Editing a schedule thirty nine minutes before its window does not warn you. It cancels the cycle.

So this is the gate, and it goes in front of every edit to a schedule carrying a pre-event, including adding or removing a machine. Read the next run time first.

Get-AzMaintenanceConfiguration -ResourceGroupName $ResourceGroup -Name $ConfigName |
    Select-Object Name, StartDateTime, Duration, RecurEvery, TimeZone

Expected: the window definition, from which you calculate the next start. If that start is under forty minutes away, stop. Do not edit, do not add a machine, do not adjust the dynamic scope. The cost of ignoring this is a cancelled run and a ring that quietly missed a month.


5. Worked example: APP01

APP01 is the legacy Windows Server 2016 application server, on premises, Arc-connected, in Ring2 with the domain controllers and the SQL server, patched last because it reboots reluctantly. It is also the machine most likely to be dark at three in the morning, and a machine that is off is skipped without complaint while the ring reports a clean cycle.

The design is a conditional wake with state capture, which survives Event Grid’s at-least-once delivery. The pre-event asks whether APP01 is reachable. If it is, do nothing and record nothing. If not, power it on and tag the machine to say you did. The post-event reads the tag and restores the previous state only if the pre-event woke it. Run either twice and the outcome is identical.

SettingValueWhy
Maintenance configurationmc-ring2-sensitiveThe event attaches to the schedule, not the machine
Pre subscriptionevt-pre-ring2-wake, Pre Maintenance EventFires at least 30 minutes before the window
Post subscriptionevt-post-ring2-restore, Post Maintenance EventFires when installation completes
Webhook Run onhwg-csj-onpremAzure cannot power on an Arc machine
Wake mechanismWhatever the site has: hypervisor start, Wake on LAN, baseboard controllerThe runbook shape is identical in all three
State tagPatchWakeState set to WokenByPreEventMakes the post-event idempotent and leaves a trail
Reboot settingReboot if requiredNever reboot here accumulates a pending restart forever
CancellationNot wired for this ringA failed wake should still let the rest of Ring2 patch
$machine = Get-AzConnectedMachine -ResourceGroupName $ResourceGroup -Name "APP01"

if ($machine.Status -eq "Connected") {
    Write-Output "APP01 already up. No action, no tag."
    return
}

# Site specific: Start-VM, a Wake on LAN packet, or a call to the BMC
& $WakeCommand

$tags = $machine.Tag
$tags["PatchWakeState"] = "WokenByPreEvent"
Update-AzConnectedMachine -ResourceGroupName $ResourceGroup -Name "APP01" -Tag $tags

$deadline = (Get-Date).AddMinutes(12)
do {
    Start-Sleep -Seconds 30
    $state = (Get-AzConnectedMachine -ResourceGroupName $ResourceGroup -Name "APP01").Status
} while ($state -ne "Connected" -and (Get-Date) -lt $deadline)

Write-Output "APP01 status at handover: $state"

Expected: the job completes with a final status of Connected, inside the twenty minute allowance and clear of the fifteen minute powered-on requirement. Failure mode: a job ending with anything other than Connected has not stopped the run, it has only told you. That catches everyone once.

The post-event is the mirror. It reads PatchWakeState, and if the value is WokenByPreEvent it issues the shutdown, clears the tag and confirms the machine has gone. If the tag is absent, APP01 was already up before any of this started and the post-event leaves it alone, which is what you want the night somebody is logged in. Confirm from the run record next morning, not the runbook output.

maintenanceresources
| where type =~ "microsoft.maintenance/applyupdates"
| where properties.maintenanceScope =~ "InGuestPatch"
| extend machineName = tostring(split(tostring(properties.resourceId), "/", 8))
| where machineName =~ "APP01"
| project name, machineName, status = tostring(properties.status),
          startTime = todatetime(properties.startDateTime)
| order by startTime desc
| take 5

Expected: the most recent runs for APP01 with a status against each, and no gap where a month should be. That last part is the real test of the event pair, and it takes two cycles to answer rather than one.


6. The end-to-end test

An alert nobody has seen fire is a hypothesis. The only way to know the whole chain works, query to identity to evaluation to action group to mailbox, is to make a run fail on purpose and watch the message arrive. Verify the target first. This is the gate in front of the only step that touches a real machine.

Get-AzMaintenanceConfiguration -ResourceGroupName $ResourceGroup -Name "mc-ring0-pilot" |
    Select-Object Name, MaintenanceScope

Search-AzGraph -Query "maintenanceresources
| where type =~ 'microsoft.maintenance/configurationassignments'
| where id has 'mc-ring0-pilot'
| project id, resourceId = tostring(properties.resourceId)"

Expected: maintenance scope InGuestPatch and exactly one machine, the Ring0 pilot. If more than one comes back, stop and fix the scope. Deliberately failing a run against a ring you thought held one server is how a test becomes an incident.

  1. On the Ring0 configuration, shorten the window to its minimum of one hour thirty minutes and set it to start shortly, respecting the fifteen minute rule for a new schedule and the forty minute rule if this ring carries a pre-event.
  2. Make sure the pilot has at least one substantial pending update, a cumulative update for preference, so the run has work that will not fit.
  3. Let the window run. The installation starts, the window ends before it finishes, and the run records a window exceeded condition.
  4. Within the rule’s evaluation period, confirm the alert fires and the action group delivers.
  5. Restore the window to its normal duration, observing the forty minute gate again if a pre-event is attached.
Search-AzGraph -Query "patchinstallationresources
| where type !has 'softwarepatches'
| extend p = parse_json(properties)
| where todatetime(p.lastModifiedDateTime) > ago(1d)
| project machine = tostring(split(id, '/', 8)),
          status = tostring(p.status),
          windowExceeded = tostring(p.maintenanceWindowExceeded),
          failed = toint(p.failedPatchCount)"

Expected: one row with maintenanceWindowExceeded true, and a fired alert with a notification in the action group’s inbox. Failure mode, and this is what you are testing for: a run record showing the failure with no alert against it means the managed identity does not hold Reader where you thought. Fix it now, while you know it is broken.


Where this goes next

The scaffolding is standing: four queries you can point anybody at, an alert proven to fire, an export that owns your evidence past thirty days, and a machine that wakes for its own window. The equivalent for the wider Arc management plane is [ARC 4.1], and where the two differ this one is later and more specific.

What remains is argument rather than build. The closing article takes the estate as it now stands, without WSUS: what moved, who holds what, where this product is the wrong answer, and what happens to the four Windows Server 2016 machines with a date on them.


Azure Update Manager
‹ Previous: [AUM 7] Operating Update Manager: The Loop After the Rings
Next: [AUM 8] The Estate After WSUS