AI vs Human Vishing: Who Gets People to Comply?

Categories: Deepfake,Published On: November 17th, 2026,
OSES™ Case Study: Voice

AI vs Human Vishing: About Half Complied Either Way

A client asked us a direct question: is a live human caller more dangerous than an AI voice agent running the same pretext? We ran an internal IT support pretext both ways and scored both on the same OSES™ ladder. Once a person was in the conversation, about half complied either way.

Human or AI, about half of the people who talked to the caller complied. What separated the two campaigns was who picked up.

Run our AI voice agent against your own help desk verification procedure.

Book Your Demo
Summarize this article with AI

Is an AI vishing agent as persuasive as a human caller?

Short answer

We could not tell them apart. On the same internal IT support pretext, 7 of 13 people who talked to a live human caller complied, and 20 of 40 who talked to our AI voice agent complied: about half either way. Complied means agreed on the call. The real gap was how many people picked up.

7 of 13

People who conversed with the live human caller and complied

20 of 40

People who conversed with the AI voice agent and complied

2 of 7

Human-call compliers who then completed the remote support workflow, in about 7 and 8 minutes

Is a skilled human on the phone a bigger threat than an autonomous agent that never gets tired? Most of the debate about AI social engineering answers that with opinion. We answered it with two scored campaigns. The headline: once someone was on the line, we found no measurable difference between a live human caller and our AI voice agent. Whether the caller was a person or a model did not show up in the result.

That finding sits squarely on what we have seen across our AI vishing red team field findings: an expected, routine request from a plausible internal function is enough. The pretext needed no deepfake and no human.

How we ran the comparison

Both campaigns used the same pretext, internal IT support, delivered by voice. Both were scored under one rule on the OSES™ ladder, so the numbers mean the same thing on both sides of the table.

FieldHuman operatorAI operator
OperatorLive human caller, no AI voiceAutonomous conversational AI voice agent
PretextInternal IT supportInternal IT support
VectorOutbound voice callOutbound call, voicemail drop, inbound callback
Follow-onEmail with a remote support link, sent only to people reached liveNone
The askAgree to a remote support sessionComply with a credential verification request

Source: Breacher.ai OSES™ engagement data. Organizations, sectors and dates withheld.

The scoring rule was identical for both. Complying with the caller's request in conversation counts as L4 Action, and verbal compliance counts. Conversing without complying (refusing, verifying, or deferring) counts as L3 Engaged. L5 Escalation is reserved for a consequence that actually followed, and downstream activity was observed in both campaigns. No credential material was captured in either. In plain terms, complied means the person agreed on the call. It does not mean credentials changed hands.

Results: no measurable difference in persuasion

MetricHuman operatorAI operator
Reached a live conversation13 of 3740 of approx. 400
Complied, over all targets7 of 3720 of approx. 400
Complied, of those who conversed7 of 13 (about half)20 of 40 (half)
Escalation (L5)0 of 7 compliant0 of 20 compliant
DEPTH4545
SPREAD30.0 (capped)8.1
OSES™ SCORE75.0, High53.1, Medium

Fisher exact test on compliance among those who conversed: p = 1.0, no statistical difference.

Read the third row first. Once someone was talking to the caller, the human got 7 of 13 to comply and the AI voice agent got 20 of 40. The test cannot tell those two results apart. That is not the same as proving they are identical: with only 13 human conversations, the human result is consistent with anything from roughly 3 in 10 to 3 in 4. So we will not publish a precise human-versus-AI rate as a benchmark. What the data supports is the plain version: about half of the people who talked to the caller complied, whether the caller was a person or our AI voice agent, on asks that were similar but not identical.

The score gap came from who picked up, not from what happened on the call.

Why the human campaign scored higher

The human campaign scored 75.0 (High) and the AI campaign scored 53.1 (Medium). Every point of that gap sits in SPREAD. DEPTH was identical at 45, because in both campaigns someone complied. SPREAD measures how much of the population went with the caller, and the human campaign got more people on the phone: 13 of 37 targets reached a live conversation, against 40 of approx. 400 for the AI campaign.

That looks like a human advantage until you check it against a second AI-operated campaign on the same IT support pretext, which reached a live person 46 of 126 times, close to the human operator's 13 of 37. One cross-check does not settle it, but it points at the target list and the organization rather than the caller. A hand-picked shortlist picks up differently from a full roster.

The tempting reading

The human scored higher, so humans are the bigger threat. This reads a reach difference, driven by who was on the list, as a persuasion difference.

What the data supports

No measurable difference in conversation. The reach gap tracked the population, and a second AI campaign picked up at close to the human rate.

What the people who held did

The most useful part of the human campaign is not the 7 who complied. It is the 5 of 13 who independently verified the caller and refused, plus 1 who deferred to another channel. There was nothing synthetic for them to catch: the caller was a real person. What stopped those calls was a verification step, not a verdict on the voice.

That is the behavior the Control Invariance Thesis says to measure. A verification step does not read the voice, so it returns the same answer whether the caller is human, cloned or generated. What a more convincing caller changes is the pressure to skip the step, which is why we measure how often the step is skipped. It is why we argue for training the procedure and then testing it rather than asking people to judge how real a caller sounds.

The other side of the ledger shows why the procedure matters. Of the 7 who complied on the call, 2 went on to complete the remote support workflow, in about 7 and 8 minutes from the email arriving. Minutes, not hours. A control that depends on someone noticing later is too slow for that window.

What this means for your program

  • Stop treating AI voice risk as a fidelity problem. The pretext needs no deepfake and no human. An expected, routine request is enough, so plan for plausibility, not realism.
  • Bind consequential actions to verification. Remote support sessions and credential requests should require a callback to a known-good number, or another channel the caller does not control, before anyone acts.
  • Measure reach and persuasion separately. DEPTH shows how far a caller got. SPREAD shows how many went with them. Reading one as the other is how the wrong conclusion gets drawn.

What this comparison does and does not show

We publish the limits alongside the result, because a score you cannot defend is not worth having.

LIMIT 01

A natural comparison, not a controlled experiment

Different organizations, different populations, run weeks apart. One campaign used a hand-picked shortlist and the other a full roster. That is why this result cannot say AI is more dangerous than humans, or the reverse.

LIMIT 02

Different asks under one rule

Agreeing to a remote session is not the same act as complying with a credential verification request. Both score L4 under one rule, but they are not identical, so treat the side-by-side as indicative.

How Breacher.ai runs AI vs human vishing tests

Breacher.ai runs orchestrated social engineering simulations under OSES™. Our autonomous AI voice agent places outbound calls, leaves voicemail, and handles the inbound callback live, so it tests the paths a real caller would use. When a client asked for a direct comparison, we ran a human operator against the same pretext and scored both on the same ladder. Findings are tied to the verification procedure that held or failed, not to the people who answered.

See how it works on the vishing simulation page, or explore the full simulation platform.

Measure your risk.
Train for what you find.
Prove it changed.

Find out whether your help desk verification procedure holds against a caller that never tires.

Book Your Demo

Frequently asked questions

We could not tell them apart. On the same internal IT support pretext, 7 of 13 people who talked to a live human caller complied, and 20 of 40 who talked to our AI voice agent complied: about half either way. Complied means agreed on the call. The real gap was how many people picked up.

JT

Jason ThatcherFounder and CEO of Breacher.ai and creator of OSES™. Fifteen years in security operations and offensive testing, previously at ZeroFox, Deepwatch, and GuidePoint Security. He builds the platform that runs orchestrated social engineering simulations, from OSINT-driven pretexts to AI voice and live deepfake video, for enterprise security teams worldwide.

Test the procedure, not the voice

In our comparison, human or AI, about half of the people who talked to the caller complied. Find out whether your verification procedure stops the rest.

Book Your Demo

Latest Posts

  • AI vs Human Vishing: Who Gets People to Comply?

  • Private Equity Phishing Simulation: How Breacher.ai Helps

  • FaceStealer: The Fake Zoom Meeting Scam Behind $400M Stolen

Table Of Contents

About the Author: Jason Thatcher

Jason Thatcher is the Founder of Breacher.ai and comes from a long career of working in the Cybersecurity Industry. His past accomplishments include winning Splunk Solution of the Year in 2022 for Security Operations.

Share this post