There is a safe way to change Conditional Access and an unsafe way, and the whole difference is what you know before you flip a policy to On. Two instruments give you that knowledge. What If answers a hypothetical about one identity. Report-only answers the real question about your whole population. The framework ships every policy in report-only for exactly this reason, and this is how you use both, read what they tell you, and promote a policy to enforcement without locking anyone out.
What If answers a question you already have
What If simulates a single sign-in. You give it an identity, a target resource, a device platform, and a client app, and it tells you which policies would apply, which would not and why, and the grant and session controls each applicable policy would demand. It is the right tool for a specific worry: you are about to enable a compliant-device policy and you want to see, for one BYOD user reaching SharePoint from an unmanaged Mac, exactly which policies bite and what they ask for.
Know its limits before you trust it, because they are the kind that mislead. What If does not perform an actual sign-in; it evaluates policy logic against the inputs you supply, and any condition you leave blank, sign-in risk, user risk, device state, location, it treats as absent. So a risk policy will look like it does not apply simply because you never told What If there was risk. It does not account for service dependencies, which means testing a Teams policy will not surface the Exchange Online policy Teams quietly relies on. Application groups like Office 365 do not match; you evaluate against a specific application ID. And only policies that are enabled or in report-only are included in a run. What If is a scalpel for a single scenario, not a survey of your estate, and treating it as the latter is how people convince themselves a change is safe when they have only tested the one case they already understood.
Report-only is the instrument that matters
Report-only is a policy state, alongside On and Off, and it is the one you live in before enforcement. A report-only policy is evaluated at every real sign-in across the tenant but never enforced: nobody is blocked, nobody is prompted. What you get instead is the truth about what the policy would have done to real people doing real work, recorded per sign-in. This is the difference between a hypothetical and a measurement, and it is why the framework ships everything report-only rather than asking you to reason your way to confidence.
You read the results in two places. On any individual sign-in event the Report-only tab shows the outcome of each report-only policy for that sign-in, and across the population the Conditional Access Insights and Reporting workbook aggregates it, which needs Entra ID P1 and a Log Analytics workspace to land the data. A newer Policy impact view snapshots the same picture over the last day, week, or month for a faster read. Every report-only outcome is one of four values, and learning to read them is the whole skill.
| Result | What it means |
|---|---|
| Success | Conditions matched and the non-interactive controls were satisfied. The policy would have let this sign-in through. |
| Failure | Conditions matched but a required control was not satisfied, a block, or a non-compliant device. The policy would have stopped this sign-in. This is the column you hunt in. |
| User action required | Conditions matched and the policy would have prompted for MFA or terms of use. In report-only nobody is actually prompted, but in production they would be. |
| Not applied | Conditions did not match. The user was excluded, the platform did not match, the location was wrong. The policy left this sign-in alone. |
The Failure and User-action-required rows are your rollout risk made visible. Before you enable a policy you go find every Failure it produced, decide whether each is a real problem or an expected block, and resolve the real ones by fixing the target, adjusting scope, or adding a governed exclusion. A week in report-only, which is Microsoft’s own stated minimum, is usually enough to surface the population’s edge cases, and the ring order in [CA 4.2] Enabling in Rings is built around exactly this cadence.
One caveat will cost you an afternoon if you do not know it. A report-only policy that requires a compliant device can still prompt a user on macOS, iOS, or Android to select a device certificate during evaluation, even though nothing is enforced, and the prompt can repeat until the device is compliant. The fix is to exclude those platforms from the report-only compliant-device policy while you are testing. And report-only cannot evaluate policies scoped to user actions, so those you validate another way.
Safe rollout is a discipline, not a toggle
The first rule is the one the framework enforces structurally: your emergency-access accounts are excluded from every policy, and you verify that at the gate before you enable anything, because a single misconfigured grant is how a tenant locks itself out of its own admin plane. The second is that exclusions are not additive protection. A user excluded from the policy you are enabling can still be caught by an overlapping one, so you test the exclusion rather than assume it. When something does go wrong you have three reversals in increasing severity: flip the policy back to report-only, exclude the affected group while you fix it, or delete it, and a deleted policy is recoverable for thirty days if you removed the wrong one. Conditional Access also has a hard ceiling of 240 policies per tenant across all states, which is worth knowing before a sprawling design walks you into it.
Underneath all of it is a habit worth building before you need it: snapshot the live policy set before every change. The policies sit behind the Microsoft Graph conditionalAccess API, and Get-MgIdentityConditionalAccessPolicy with Policy.Read.All exports the entire set in one call, which is both your backup and your diff. The framework already treats its repository as the source of truth and ships policies as generated JSON; the tenant export is the other half, the record of what is actually live, so a bad change is one restore away rather than an afternoon of reconstruction from memory. Protecting the policy surface itself with protected actions, so editing a Conditional Access policy demands fresh phishing-resistant authentication, closes the loop. What If for the one scenario you are worried about, report-only for the population you cannot predict, and an export before you touch anything: do those three and enabling a Conditional Access policy stops being an act of faith.
Conditional Access
‹ Previous: [CA 2] Conditional Access Frameworks and Policy Design
Next: [CA 3] Named Locations, Risk Policies, and Identity Protection ›
