A tool to assess historical impact of report-only policies
Note
Development on this solution is on hold until some other projects are finished. Plan to revisit in the second half of 2025.
This solution uses Microsoft Graph to read Conditional Access policies and evaluate the impact it would have had over the last 30 days. The initial focus is on MFA and device attributes, and some things such as filter for apps will likely be too complex to ever add to this solution.
It is highly recommended that we ingest sign-in logs to Log Analytics and use the Insights and Reporting workbook to evaluate impact policies had or would have had in our environment. This will still be the ideal way to perform potential impact assessment, but it has two limitations - requires Log Analytics ingestion (not set up yet, cost is a factor) and any policy changes cannot be retroactively assessed.
Put another way, the hope here is that we can create policies, run a tool to assess impact, then make tweaks to the policy, run the tool again, and continue this process until we are happy - no KQL or Graph knowledge needed :)