When leadership asks "are we protected?", the answer most IT teams give is some version of: "yes, MFA is on, Conditional Access is configured." That answer is usually true on the surface and rarely true in practice. The Conditional Access blade in the Entra admin center shows a green checkmark next to "Configured." The Sign-in Logs show MFA prompts firing. Everything looks fine.
The gap between a Conditional Access deployment that looks configured and one that is actually protecting users is the most consistent finding from the tenant audits we run at NeoDefender. The features are not exotic. The fixes are usually not expensive. What is missing is the second look that checks whether the policies in place are doing the work they appear to be doing.
These are the patterns we see most often. If your team has deployed Conditional Access and is not entirely sure what it is currently catching or missing, this list is for you.
Mistake 1: Policies scoped to "Selected users" from when the tenant was smaller
Many Conditional Access deployments started cautiously, scoped to a hand-picked list of users in the early days of the tenant. The cautious approach was correct at the time. The problem is that the list never grew.
We routinely audit tenants where the baseline MFA policy was assigned to 50 users in 2022 and the tenant has since grown to 200 employees. The 150 newer users are not protected by that policy at all. They sign in with whatever Microsoft's defaults provide, which in most cases is not equivalent to MFA enforcement.
The remediation is to scope baseline policies to "All users" with explicit exclusions for break-glass accounts and documented service principals, and to move user-by-user rollouts into Microsoft's report-only mode rather than into user assignment scoping.
Mistake 2: Administrators excluded from MFA "to be safe"
A surprising number of tenants have a policy structure that reads, in effect, "Require MFA for all users except administrators." The original intent was usually to prevent admins from being locked out during deployment. The result is that administrators, the highest-value accounts in the tenant, are the only users not enforced.
Attackers look for tenants with this exact configuration. Compromising a non-MFA administrator account is the fastest path to full tenant takeover, and we covered the adjacent attack pattern in our post on token theft and why MFA alone is not enough.
Administrators should have stricter Conditional Access requirements than regular users, not looser ones. Break-glass accounts are the exception. Everyone else with elevated privileges should be on phishing-resistant MFA at minimum.
Mistake 3: Legacy authentication still allowed
Legacy authentication protocols (basic auth for IMAP, POP, SMTP, older Exchange clients) do not support modern MFA. They are the single most common bypass attackers use, and they remain enabled by default in many tenants we audit, including tenants where the rest of the Conditional Access configuration looks reasonable.
The reason this gap survives is usually that one or two legitimate services in the tenant still depend on legacy auth, and the team decided to "deal with it later." Later never comes. Meanwhile, every credential in the tenant is reachable by an attacker willing to brute-force or password-spray the legacy endpoints.
The remediation is to identify everything currently using legacy auth (Sign-in Logs filtered by client app reveal this clearly), migrate it to modern authentication or service-specific app passwords, and then enforce the block. Doing this in report-only mode for a week before enforcement catches the surprises.
Mistake 4: "Configured" policies that are still in report-only mode
Conditional Access has three states for each policy: On, Off, and Report-only. Report-only evaluates the policy and writes the result to Sign-in Logs but does not enforce. It is the right way to test a policy before going live.
The mistake is leaving policies in Report-only mode indefinitely. We have audited tenants where critical policies (block legacy auth, require compliant device for sensitive resources, restrict guest access) had been in Report-only mode for over a year. They generated logs nobody read. They provided no protection. From the admin center, they looked active. From the attacker's perspective, they did not exist.
Report-only mode is a testing tool, not a parking space. Every Report-only policy should have a documented review date and an enforcement decision.
Mistake 5: Guest users inheriting employee policies (or worse, none at all)
When a Conditional Access policy applies to "All users," it includes guests by default. Guests are external collaborators, vendors, contractors, and B2B partners. They may not have your MFA configuration, FIDO2 keys, or compliant devices.
We see two failure modes here. The first: guests are included in policies designed for employees, which can block their legitimate access (the partner who cannot reach the shared SharePoint site). The second, more common: guests are explicitly excluded from policies to avoid that friction, which means they sign in with no enforcement at all.
Guest accounts need their own Conditional Access structure: shorter session lifetimes, MFA every sign-in (not just first time), no access to administrative portals, and explicit allowlisting of which applications they can reach. They should not inherit employee baselines, and they should not be a free pass.
Mistake 6: No device compliance enforcement, even when Intune is deployed
A common audit finding: Intune is deployed, devices are enrolled, compliance policies exist. And no Conditional Access policy actually references device compliance. Users sign in successfully from any device, managed or unmanaged, because nothing in the access path checks the compliance status.
This is paperwork without consequence. The fix is straightforward: a Conditional Access policy that requires "compliant device" or "hybrid Azure AD joined device" for access to sensitive resources, with the policy referencing the compliance state Intune is already tracking. We covered the broader picture of device management posture in our post on from patching to risk management.
Mistake 7: No risk-based policies, even with Entra ID P2 licenses
Tenants with Microsoft 365 Business Premium or E3+Entra ID P2 add-on have access to Identity Protection risk signals (leaked credentials, anomalous sign-in properties, atypical travel, suspicious tokens). These signals are useful only if a Conditional Access policy acts on them.
In tenants we audit, we find Identity Protection generating risk signals daily, and zero policies configured to act on those signals. Microsoft's newer "Require Risk Remediation" condition (which we covered in our post on the newest Microsoft 365 security features) makes this even easier to deploy: instead of blocking risky sign-ins outright, the system can require users to remediate the risk before continuing. The licenses are paid for. The signals are flowing. The policy is the missing step.
Mistake 8: No documentation, no review cadence
The last mistake is structural rather than technical. Conditional Access policies accumulate over time. A typical mid-stage tenant we audit has 15-25 policies, of which several are duplicates, some are in stale "Test" state nobody remembers creating, and a few have exclusions for individual user accounts of employees who left the company years ago.
A healthy Conditional Access estate has:
- A naming convention that makes intent obvious from the policy name.
- A description field populated on every policy explaining what it does and why.
- A quarterly review to identify duplicates, orphaned exclusions, and policies no longer matching current business needs.
- A change log maintained outside of the tenant so that when something behaves unexpectedly, the team can trace which change introduced it.
How to tell if your Conditional Access is doing the work
Three quick checks, each takes about 15 minutes:
Sign-in Logs, last 30 days, filtered by Conditional Access result. If "Not applied" dominates the results, your policies are scoped too narrowly. If "Report-only" dominates, your enforcement story is theoretical.
Policy inventory export. Count how many policies you have. For each one, write down what it does in a single sentence. If you cannot, the policy probably is not doing what its name implies.
Identity Protection risk report. If risky users and risky sign-ins are accumulating with no remediation activity, your Entra ID P2 license is generating data that nobody is acting on.
The findings from those three checks usually surface 80% of what an external audit would find. The remediation is usually less disruptive than people fear, provided break-glass accounts are in place and the changes are sequenced correctly.
This is the work we do as part of our Identity Protection service and within our Reality Check audits. We have walked through Conditional Access estates that grew across multiple IT teams over five years, and the patterns are predictable.
If your tenant has Conditional Access configured but nobody has reviewed it in the last twelve months, schedule a call with the NeoDefender team. The features you have already paid for are usually closer to delivering real protection than the configuration suggests.
