There is a version of deepfake testing that has become the industry default, and it does not work very well. A cloned voice places a call. Someone answers or does not. A number gets written down. The report says a percentage of the population engaged with a synthetic caller, and everyone agrees the technology is impressive.

We run these engagements for a living, and the single most useful thing three years of them has taught us is that the outbound call is the weakest leg of the sequence. It is the leg an adversary has the least confidence in, the leg most likely to be refused, and the leg most likely to be reported. It is also, almost universally, the only leg anybody simulates.

The consequences we see land on the inbound. A voicemail goes out. Some hours later the target picks up their own phone and dials the number in it. That return call is where the credential goes across. In our red team work, the return path is where the majority of credential handovers happen, and it is the path with the least procedural protection standing in front of it.

This post is the argument for why orchestration is not a feature of a deepfake phishing simulation. It is the thing that determines what the simulation is capable of observing at all.

The Voice Channel Produces Action That Nothing Records

Before the callback argument, the baseline. Voice campaigns produce consequential outcomes in populations where an email-first tool would have reported an empty result, because there was no link anywhere in the sequence for it to count.

Source: Breacher.ai OSES™ engagement outcomes, published in the OSES Risk Index. Detection figures are measured across engagements covering more than one thousand individual targets. Reporting is organizational and sector level only.

Those figures describe the channel. They do not tell you where inside the channel the failure sits, and that is the question orchestration exists to answer.

The Sequence We Actually Run

OSES™, Orchestrated Social Engineering Simulations™, sequences each contact against the state left by the one before it. Below is the IT support variant, which is the one that produces our most serious findings. Note where the featured card sits.

Why the Return Leg Is Where It Breaks

Three mechanisms, and they compound. None of them is about how good the audio was.

Self-initiation is mistaken for authentication. Every instinct a user has about verifying a caller is built around receiving a call. Dialling a number yourself feels like the verification step, because in the user's mental model it is the same motion as looking up the help desk and calling them. The distinction between a number you looked up and a number you were given collapses the moment several hours have passed and the transcript is sitting in the phone.

Absence of pressure suppresses the report, not the risk. This is counterintuitive and it is the finding people push back on. A high-pressure call leaves a mark. The target remembers being rushed, mentions it to a colleague, and sometimes reports it. A calm return call on the target's own schedule leaves no mark at all, because nothing about it registered as an event. The handover happens and then nothing happens, which is the worst possible outcome for the organization and the reason we treat report rate on the inbound leg as a separate measure.

The path is uncovered, not merely unheld. Read your own verification procedures and count how many are written for an inbound contact. Nearly all of them. A self-initiated call is treated as self-authenticating by default, so there is no requirement to skip, no exception to grant, and no step for anyone to fail. This is not a discipline problem in your people. It is a hole in the control surface, and it is invisible to any test that never traverses it.

This Is a Coverage Problem, Not an Adherence Problem

The distinction is worth being precise about, because the remediation is completely different depending on which one you have.

Adherence is whether a documented verification requirement was executed when it applied. You improve adherence with training, with a lower-friction procedure, and with removing the exception path that lets a helpful person skip it. Coverage is whether a verification requirement exists for a given consequential action in the first place. You cannot train your way to coverage. Someone has to write the procedure.

Most organizations have a verification requirement for outbound payments and nothing whatsoever for credential resets, vendor banking detail changes, multi-factor enrollment, privileged access grants, software installation on request, or a support conversation that arrives over Teams. The consequential actions worth listing are short and every one of them should have a named requirement attached to it. On the return leg, the coverage gap is close to total.

It is also worth stating the bound honestly. Better synthetic media does not defeat a callback procedure directly. A callback to a number the organization already holds returns the same result against a crude clone and against a perfect one, because the procedure never listens to the audio. What improves with generation quality is the pressure applied against the procedure, which raises the rate at which someone waives it. We track that as exception rate rather than claiming the control is untouchable, because the stronger claim is the one somebody will eventually falsify in front of us.

What Each Architecture Can Observe

Simulation platforms fall into three bands. The useful question is not which channels a platform sends on. It is what the architecture is able to see.

The uncomfortable middle band is the one to watch. Adding a voice call to an email tool is real progress and it is better than not testing voice. But partial simulation produces partial confidence, and a workforce that passed a standalone vishing test can leave a security leader believing the organization is covered on a sequence it has never been tested against.

What an Orchestrated Simulation Lets You Measure

Click rate does not survive contact with this channel, and neither does any single headline number. These are the measures we report, and the two at the top are the ones most organizations have never calculated.

• Coverage rate. Consequential actions that carry a defined verification requirement, over consequential actions identified. This can be assessed before a single call is placed, which makes it the cheapest place to start.
• Process hold rate. Verification executed, over verification required. The headline figure, and the one that replaces click rate.
• Exception rate. Verification consciously waived or talked past, over verification required. This is the number that moves as generation quality improves, so it is the one to trend over time.
• Callback verification rate. On the return leg specifically, how often the target confirmed the number against the directory before acting. Reported separately, because a blended figure hides the gap.
• Report rate and time to first report. Measured per leg. A refusal that nobody escalated leaves the next target facing the same caller with no warning in front of them.
• Action rate, kept as a secondary figure. It moves with scenario difficulty and it does not name the control that failed, so it belongs in the report rather than at the top of it.

Findings are named against procedures rather than individuals, and results roll up to the organization. A per-user result is useful for deciding who receives which training module. It is not an outcome metric, because a score on a person does not tell you whether the wire went out.

What We Would Do First

Three things, none of which requires a purchase order, in this order.

List your consequential actions and mark the uncovered ones. Outbound payment, vendor banking detail change, credential reset, multi-factor enrollment or reset, privileged access grant, software installation on request, data export, physical access grant. For each one, write down whether a verification requirement exists, whether it is independent of the channel the request arrived on, and whether the person who will be asked at four on a Friday can actually reach it. Most teams find this exercise takes an afternoon and produces the most useful page in their programme.

Extend one rule to cover self-initiated contact. The standing rule is usually written as a rule about incoming calls. Rewrite it so it covers the direction that is currently exempt: a number obtained from a voicemail, a text, or a caller is not a verified number, and the only verified number is one looked up independently. That single amendment closes the largest uncovered path we find.

Then run the return leg, not just the call. Place the calls, leave the voicemails, answer the callbacks, and follow through into the reset or the approval. Report the coverage rate, the process hold rate, and the callback verification rate separately. The gap between what people say they would do and what they do on a call they placed themselves is the entire finding.

If you want the channel detail behind this, the voice mechanics are covered under AI vishing simulation, and the full multi-channel engagement under deepfake phishing simulation.

Frequently Asked Questions