We run authorized voice engagements against enterprise organizations for a living, which means we also read a lot of other people's assessment reports. The common failure is not that the calls were unrealistic. It is that the assessment was scoped against a story rather than against a set of actions, so the report describes what happened on the phone and cannot tell the buyer which procedure to change on Monday.

A voice phishing assessment is worth its cost when it answers a narrow question: if a credible caller asks for one of the things that would actually hurt us, does the verification step happen. Everything else in the engagement is machinery in service of that question. This is the scoping and reading guide we give buyers, including the parts that make our own findings less flattering.

What a Voice Phishing Assessment Is, and What It Is Not

The term covers three different products in this market, and the differences matter more than the shared label.

It is not a questionnaire. A maturity survey records what a policy says and what people believe they would do. Every organization we have tested had a callback policy on paper. The gap between the policy and the phone call is the entire subject of the assessment, so a method that reads the policy cannot measure it.

It is not an awareness exercise with audio bolted on. Playing a recorded message at a population and counting who stayed on the line measures availability. A verification procedure is a conversation, and it can only be tested by something that answers a question, tolerates being challenged, and handles the callback when one is requested.

It is an authorized measurement of a control. Calls run under signed authorization against your own organization, targeting the procedures that stand between a request and a consequential action. The finding is which procedure held, which one was skipped, and which one never existed. The mechanics of how the calls are delivered are covered on our AI vishing simulation page. This piece is about scoping and reading the result.

The Five Decisions Made Before Anyone Dials

These are decided in scoping, and they determine whether the report is actionable. Getting them wrong is not recoverable by running better calls.

Why an Assessment Scored on Detection Tells You Nothing You Can Fix

A large share of voice assessments still report a version of how many people noticed. It reads well and it does not survive contact with the next model release.

Detection is a decaying control. Its effectiveness is a function of how good the synthetic audio is, and generation quality improves continuously and cheaply while human hearing does not improve at all. A finding expressed as a detection rate has a shelf life set by the adversary rather than by you.

Procedural verification is invariant to the same variable. A callback to a number taken from your own directory returns the same result against a crude clone and against a perfect one. The procedure never listens to the audio. That is the whole reason it holds.

The honest bound on that claim. Better synthetic media does not defeat a verification requirement directly. It raises the pressure applied against it, which shows up as a rise in the exception rate, meaning the number of times someone consciously waived a step that they knew existed. That is a measurable variable and we report it separately, because claiming perfect invariance is the kind of statement someone eventually falsifies in front of you.

None of this removes people from the measurement. Procedures are executed by people under pressure, so the assessment is still measuring human behavior. It has simply moved from a variable that cannot be trained to one that can. We covered how the same argument plays out at the service desk in our breakdown of IT support impersonation.

Figures from Breacher.ai OSES™ engagement data, published in the OSES Risk Index.

What the Report Has to Contain

Read a sample report before you sign anything. These are the elements that make findings usable, and their absence is the most common reason an assessment produces a slide instead of a remediation plan.

• Coverage, stated as a figure. How many of your consequential actions carry a verification requirement, and which ones do not. The uncovered paths are the remediation program, and they can be sized before a single call is placed.
• Action rate. How often the call produced the outcome it asked for: a reset completed, a code read back, an approval granted, a payment detail changed. This is the closest equivalent to a click and it is a far more serious event.
• Process hold rate. How often the documented verification procedure was executed end to end under pressure. A low action rate with a low hold rate means you were lucky rather than protected, and the report should say so in those words.
• Exception rate, separated from omission. Verification not attempted and verification attempted then waived are different failures with different fixes. A report that collapses them into one number has thrown away the distinction that decides your remediation.
• Report rate and time to first report. Including the calls that were refused and never escalated. A person who resisted and said nothing leaves the next person facing the same caller with no warning.
• Findings written against procedures, not people. Organizational and process level reporting, with no named individuals and no departmental leaderboards. Per-user results are useful for routing training to the path that failed. They are not the outcome metric, because a score on a person does not tell you whether the payment goes out.
• A peer position, not just your own trend line. An internal comparison tells you whether you improved against yourself, which is a low bar when you do not know where you started relative to comparable organizations.

Questions Worth Asking Before You Sign

Ask these of any vendor, including us. The answers separate an assessment from a demonstration.

The callback question is the one that most reliably separates vendors. Real voice fraud converts on the return call, because a number the target dialed themselves feels verified before anyone speaks. An assessment that cannot answer the phone has skipped the part where the loss usually happens.

What We Would Do First

If you have one week and no budget, do these three things in this order. None of them requires a vendor.

Write the consequential action list. One page. Every action in your organization that moves money, grants access, changes a payment destination, or discloses data on request. Do it with finance, the service desk, and identity in the room, because the list is always longer than any one of them expects.

Mark which ones have a verification requirement. Not a policy that mentions verification in general terms, a specific requirement attached to that specific action, reachable by the person who performs it. Count the marks. That fraction is your coverage, and for most organizations the first calculation of it is uncomfortable.

Then run the calls against the uncovered paths first. Testing a well-covered wire process proves a control you already have. The value is in the paths where nothing stands between a plausible request and a consequential action, because those are the ones an adversary reaches without a fight.

Frequently Asked Questions