Most organizations have tested their people. Very few have tested their process. The distinction sounds like semantics until you watch both happen in the same week, against the same population, and get two completely different numbers.

We build and run orchestrated social engineering simulations as paid engagements against enterprise organizations. That is the whole business, and it is the reason this piece is written in the first person rather than assembled from threat reporting. What follows is what orchestration means operationally, what an orchestration platform has to be able to do before the word is honest, and what the output looks like when it is done properly.

What an orchestrated social engineering simulation actually is

An orchestrated social engineering simulation is a controlled, authorized exercise in which synthetic voice, video and messaging are coordinated into a single campaign, and the response is observed from first contact through to a consequential action. A consequential action is the thing that would have cost you something: an outbound payment, a vendor banking-detail change, a credential reset, an MFA enrollment, a privileged access grant, a data export.

The defining property is the sequence. Each artifact references the one before it, and each one crosses a different control boundary. A target who receives a single unexpected message evaluates it cold. A target who receives a voicemail, then a text confirming the voicemail, then a video call from the person in the voicemail is not evaluating anything by the third step. They are continuing something already in progress.

Each step borrows credibility from the one before it. The sequence, not the artifact, is what a single-channel test cannot reproduce.

Five capabilities an orchestration platform has to have

These five are the capabilities that decide whether a product can actually run the sequence above, and every one of them is a question you can ask in a demo and watch someone either answer or deflect.

Why a single-channel send under-reports your risk

It tests the one channel your controls already cover. The email gateway is the most mature filter in the enterprise. A program that only sends mail is sampling the best-defended path and reporting the result as an organizational number. Meanwhile the phone, the meeting invite, the chat message and the help desk queue have nothing in front of them.

It terminates before the expensive part. A click is where a phishing test ends. It is where an incident starts. Everything that determines whether the click costs anything, the callback, the approval, the reset, the second signature, happens after the point at which most programs stop recording.

It produces a metric that cannot be acted on. A click rate tells you a number went up or down against yourself. It does not tell you which verification requirement was missing, which process had no defined step, or which of those two failures you are actually looking at. Those are different remediations, and a send cannot distinguish them.

What the sequence produces that a send does not

Figures are from published Breacher.ai engagement reports. Each carries the population it was measured against, and none of them should be read as an industry rate.

The middle figure is the one people quote back to us, and it is also the one we are careful about. It says detection did not work. It does not say training is worthless, and we do not make that claim. It says that a control whose effectiveness is set by the quality of the adversary's generation model is a control with a declining value, because generation quality improves continuously and human perception does not improve at all.

The first figure is the one that should change what you buy. More than one in five people in that population rang a spoofed number back and talked to a machine, having already decided the counterparty was real. That path does not exist in an email test. There is no version of a send that finds it.

What we report, and what we deliberately do not

Verification is the control that holds, because it does not read the artifact. A callback placed to a number from your own directory returns the same result against a crude fake and a perfect one. That property is why the reporting is built around the process rather than the person.

• Action rate. The share of targets who performed the consequential action. This is the headline, and it replaces click rate.
• Verification rate. The share who checked through an independent channel before acting. This is the control actually working, and it is the number that should move after remediation.
• Report rate and time to first report. A person who felt something was wrong and hung up without reporting is not a success. It is an adversary calling the next name on the list with nobody watching.
• Where the sequence survived. Which step crossed which control boundary untouched, so remediation is sequenced by layer rather than by scenario.
• A peer vertical benchmark. Your own last quarter is a low bar when you do not know where you started relative to comparable organizations. The Social Engineering Risk Index is where the sector medians are published.

This is the practical case for orchestration, and it is a measurement case rather than a technology one. An orchestrated simulation measures risk far better than a send can, because it puts the request in front of the process that would actually have to stop it. A single-channel test can only report that somebody interacted with something. A sequence carries the request down the path a real one would take, through the channel it would arrive on, to the person who holds the authority, and then records what the process did with it. The output is not a bigger number or a more alarming one. It is a number that names a control.

That changes what the report is for. A click rate gives you a trend line against yourself. An action rate paired with a verification rate gives you a specific failed step, on a specific path, with a defined owner, which is the difference between a finding somebody can act on this quarter and a metric that gets presented and then filed.

Six questions for any orchestration vendor

Including us. These are the questions that separate a product that runs sequences from a product that sends mail and has added the word orchestrated to its homepage.

We wrote up the single most reliable sequence we run, help desk impersonation, in detail in the IT support impersonation breakdown. The engine and the capability list behind all of this live on the OSES™ platform page, and the synthetic media layer is detailed under deepfake simulation.

What we would do first, at no cost

Before running anything, list the consequential actions in your organization. Outbound payment, vendor banking-detail change, credential reset, MFA enrollment or reset, privileged access grant, software installation on request, data export, physical access grant. That list is short, it is specific to you, and almost nobody has written it down.

Then build a risk register from it. One row per consequential action, with these columns filled in:

• Who can perform it. The roles that actually hold the authority, not the policy owner. It is usually more people than the policy suggests.
• Which channels a request can arrive on. Email, phone, Teams or Slack, a ticket, a meeting, someone at a desk. Every channel on that list is a path an adversary can choose.
• Whether a defined verification requirement exists. Written down, not assumed. If two people describe it differently, it does not exist.
• Whether that requirement is independent of the channel. A callback to a number supplied during the request is not verification, it is the same channel wearing a different hat.
• Whether the person under pressure can reach it. If the directory number is three clicks into an intranet the caller is telling them not to open, the control is theoretical.
• What it costs when it goes wrong. Money moved, access granted, records disclosed. This is what sequences the remediation, not the likelihood.

Then map each row to the layers that should have caught it: which technical control should see the request, which process should stop it, and which role is expected to challenge it. Where nobody can name all three, that is the finding, and you found it without running anything.

That is a structural assessment. It requires no simulation, no vendor and no budget, and in our experience most organizations have never done it. The answer is usually that a verification requirement exists for wire transfers and for almost nothing else. If that is what your register shows, you have the remediation program before a single test has run, and you know exactly which rows a simulation should be pointed at first.

Frequently Asked Questions