Many deepfake detection vendors quote an accuracy figure around 99%. Almost none are comparable, because each is computed on a test set and a threshold the vendor chose.
Synthetic media has moved enterprise fraud from “spot the tell” to “verify the person”. In the Arup case, everyone on the call recognised the CFO’s face and nobody verified it, and the company wired US$25.6 million to fraudsters. That is the synthetic-media trust gap in one story: the face looked right, so the controls went quiet.
The chain: judge the vendor’s claim, decide build or buy, translate the residual risk into board language, and recognise the human layer as the part no tool replaces.
How do I evaluate whether a detection vendor’s accuracy claims are credible, and what should I look for and ask during procurement?
Treat any self-reported accuracy figure as a lab claim, not proof. Ask which threat the system addresses: presentation attack detection (PAD), injection attack detection (IAD), or deepfake detection. Then ask which generators and compression conditions it was tested on, and its false-positive rate at your prevalence. Run it on your own held-out media against a pinned model version.
Most scorecards ask for “liveness” and stop there, so you defend the camera while the attack is a virtual feed in the pipeline. The question is not on the form, so the answer is not in the contract. PAD defends the camera, IAD the pipeline, deepfake detection the image; a “99% liveness” claim says nothing about synthetic or injected media.
Whatever the headline, it is usually a lab number. Vendor-neutral testing finds cross-generator generalisation drops accuracy by 12 to 30% on unseen generators, and top systems degrade 15 to 25% more under compression and re-encoding.
Demand compression-resilience benchmarking, held-out generators, demographic bias testing, and the diffusion and GAN mix. Public benchmarks freeze a set of generators in time, so a leaderboard score is an upper bound, not an estimate. The strongest signal is how the vendor behaves on your data: refusing it, or claiming no interesting failures, means the edges are unmapped. Hard questions should make a vendor more specific, not more fluent; specific is engineering, fluent is sales.
Check the output: confidence scoring over a binary verdict, so you can triage, and multimodal detection across audio, video and image. Sumsub tracked a four-fold rise in deepfake fraud year on year, and “Frankenstein” identities fuse stolen data with AI biometrics to slip past KYC. Reality Defender and Pindrop are capability categories, not recommendations. When a vendor touts provenance support, ask what it actually proves.
How you answer those questions decides whether you build or buy.
How do I decide whether to build or buy synthetic-media detection?
Decide on control, data sensitivity and pace; cost is the trailing variable. Build where data custody, custom thresholds or privacy-preserving verification are strategically differentiating. Buy where modality breadth, telemetry and support matter more as a commodity layer. Keep an abstraction layer so you can swap either side later.
A spreadsheet misses the variables: control and pace. Building on open-source frameworks gives you ownership of thresholds, data residency and the audit trail, and lets you adopt zero-retention biometrics with zero-knowledge proofs, aligned to NIST SP 800-63-4 at IAL2 and GDPR, and ISO 20022 for payment-blockade designs. A zero-retention architecture processes biometric data in local memory for a single match, then deletes it.
The catch is the retraining burden now sits with you. Buying flips it: APIs and managed platforms give speed and a real-world corpus moat, like Pindrop’s years of telephony audio, but your data leaves the organisation and lock-in risk rises.
Match the mode. A real-time factor is processing time divided by media duration, and you need it below 1.0 for live calls and meetings; batch forensic analysis suits after-the-fact investigations. Treat this the way platform teams treat moderation, as infrastructure you own and instrument rather than a product you bolt on.
Once you own it, it becomes a risk you report upward.
How should you frame the synthetic-media trust gap for a board or CEO?
Translate the trust gap into risk, liability and budget language. Lead with the financial consequence, then map the compliance obligations driving spend and state the residual risk plainly, because detection cannot be guaranteed.
Give the board what happens when the detector is wrong. This is why boards are asking about synthetic media. Arup wired US$25.6 million out of Hong Kong because a finance employee trusted a video call full of deepfakes, and Sumsub’s four-fold rise in deepfake fraud is the trend line.
Then map the budget to obligations, not just threats. The compliance obligations driving defence budgets include the EU AI Act‘s transparency rules, the UK ECCTA “failure to prevent fraud” offence, GDPR, and SEC and SOX disclosure duties. Liability is already settled: your vendor’s compliance posture is now your compliance posture, the deployer carries the risk of false outputs.
State the residual risk plainly. No detection system in vendor-neutral evaluation held above 90% across all modalities, so a detector is not an oracle. Adequate defence is defence in depth: provenance first, detection as assistive triage, a human in the loop, and residual risk documented in a risk register with materiality thresholds, assurance language, and a model-drift SLA. At realistic prevalence, where fakes are a fraction of one percent of interactions, a high-accuracy detector still produces mostly false alarms, so budget for triage, not an oracle.
That triage is people, which is where the human layer does its work.
Detection technology vs employee training for deepfake defence: which actually reduces risk?
You need both, but training closes the gap detection cannot. Detection misses novel, out-of-distribution fakes, and the human element is present in 62% of breaches, so deepfake phishing simulation modelled on real executive impersonation builds the out-of-band verification habit tooling cannot.
That Verizon DBIR figure is the anchor: the human is the vector. A detector decays against unseen generators, so an attacker routes around it to the person on the call. Under time pressure, human deepfake detection collapses towards a coin flip for synthetic video.
The trade-off is tools catch volume, training catches novelty. The human-layer risks are automation bias and authority bias, trusting the detector or the “executive’s” voice. The reliable red flag is behavioural: a request that bypasses approval, manufactures urgency or resists out-of-band verification warrants scepticism. Simulation, rather than an annual awareness video, is what builds the habit; measure behavioural readiness, detection latency, verification lag and fail rate. Frame it as a budget decision: how much of the gap each layer closes.
Conclusion
You cannot close the trust gap with any single tool. The discipline is stating residual risk honestly rather than promising it away.
Synthetic-media defence becomes an owned residual risk rather than a purchased detector. Provenance support is the complementary half of defence: it proves a file is authentic when credentials are present, and its absence proves nothing.
Regulation turns each decision into board accountability. You manage the trust gap rather than close it, and the last line of defence is a person who verifies before they trust.
Frequently Asked Questions
What is the difference between PAD, IAD and deepfake detection?
They address different threats and are not interchangeable. Presentation attack detection (PAD), or liveness, checks whether a real person is physically present, catching masks, photos and replayed video. Injection attack detection (IAD) catches a virtual camera feed being pushed into the pipeline, while deepfake detection looks for synthetic or manipulated media. A “99% liveness” claim says nothing about injected or AI-generated content, so buy against the threat you actually face.
Why does a detector rated at 99% accuracy still generate mostly false alarms?
Because of base rates. When genuine fraud is rare, even a highly accurate detector flags mostly innocent cases. If fakes are one in ten thousand interactions, a detector with a 1% false-positive rate will produce far more false alarms than real catches. That is why you budget for human triage rather than an oracle, and why false-positive rate at your prevalence matters more than a headline accuracy figure.
What is a synthetic or “Frankenstein” identity?
It is a fake identity that fuses stolen real data with AI-generated biometrics. Fraudsters combine genuine details, such as a real person’s name and address, with synthetic faces and voices so the identity passes document and liveness checks. Because part of the record is authentic, it can slip past KYC systems that verify data points individually rather than checking whether the whole identity is internally consistent.
What is model drift, and why does it matter after deployment?
Model drift is the decline in a detector’s accuracy as the media it encounters moves away from its training data. New generators, fresh compression settings and evolving synthesis techniques push inputs out of distribution, so a detector that scored 99% in testing can lose 12 to 30% of its accuracy in the field. That is why contracts should include drift monitoring and retraining terms, with performance reported as a residual risk to the board.
What is content provenance, and can it replace detection?
Provenance records where a piece of media came from and whether it has been altered, typically through signed metadata such as C2PA. It is the complementary half of defence, not a replacement. Provenance proves a file is authentic when credentials are present, but its absence proves nothing, because most media carries none. Use it as your first layer, then detection as assistive triage for everything provenance cannot vouch for.
Do deepfake detectors work on voice calls and audio, not just video?
Yes, but coverage is uneven, so check it explicitly. Telephony has its own detection category, and some vendors hold deep corpora of real call audio, which is a genuine moat. Still, a video-fraud detector is not automatically strong on voice, and vice versa. Ask which modalities a system actually covers, whether it analyses audio, video and image together, and how it performs on the channel you rely on, such as live phone banking.
What does a real-time factor below 1.0 mean, and when does it matter?
Real-time factor is processing time divided by media duration. A factor below 1.0 means the system analyses faster than playback, so it can score a video call as it happens. You need that for live verification, such as a video call authorising a payment. For after-the-fact investigations, batch forensic analysis is fine and often deeper, so match the processing mode to the use case rather than paying for speed you do not need.
What is out-of-band verification, and why does it matter more than detection?
Out-of-band verification means confirming a request through a separate, trusted channel rather than the one it arrived on. If an executive asks for a payment on a video call, you call them back on a number you already hold. It matters because detection cannot be guaranteed, so a novel fake can pass every tool and reach a person. The habit of verifying through a second channel is the control that closes the remaining gap.
Who is liable when a deepfake fraud succeeds?
Liability is shifting toward the organisation that failed to prevent it, not just the fraudster. The UK’s ECCTA “failure to prevent fraud” offence and disclosure duties under SEC and SOX rules mean boards can be held accountable for inadequate controls. That is why synthetic-media defence belongs in the risk register, with materiality thresholds and assurance language, rather than sitting with the security team as a technical matter.
Are deepfakes illegal?
Not as a technology, but their malicious use increasingly is. Creating synthetic media is legal in most jurisdictions, while using it to defraud, impersonate or spread harmful disinformation is not. Regulation is tightening around disclosure, not prohibition: the EU AI Act requires transparency for AI-generated content, and other regimes target non-consensual and fraudulent use. Treat compliance as a disclosure and record-keeping obligation rather than a ban you can rely on.
How often should we re-test a detection vendor after signing?
At least every six months, and after any major new generator release. Vendor accuracy decays as the adversary advances, so a one-off procurement test tells you little a year later. Run the vendor’s model on your own held-out media against a pinned version, track its false-positive and false-negative rates over time, and hold them to a drift SLA. Re-benchmarking is how you turn a lab number into a managed, board-reportable control.
Can open-source detection tools replace a commercial vendor?
Sometimes for a narrow threat, rarely for broad coverage. Open-source frameworks give you control over thresholds, data residency and auditability, which suits privacy-sensitive, high-stakes verification. But they do not come with the vendor’s real-world corpus, telemetry or support, and you carry the full retraining burden against an advancing arms race. Build the differentiating parts, buy the commodity detection layer, and keep an abstraction layer so you can swap either side.
