The voice can sound familiar before the listener has time to ask why.
The accent is close. The rhythm is right. A pause lands where a real person might pause. The message arrives with urgency attached to it: something has happened, there is little time to explain, and a decision has to be made now.
For most of the history of recorded communication, recognising a familiar face or voice has carried enormous psychological weight. It feels like direct evidence of identity.
Artificial intelligence is making that assumption less dependable.
The central problem is not that every synthetic recording is flawless. Many are not. The problem is that fraud, misinformation and harassment do not always require technical perfection. They require something convincing enough, shown to the right person, at the right moment.
The fraud that looked like a company meeting
One of the clearest warnings came from Hong Kong.
Documented case
HK$200 million transferred after a fabricated video conference
Hong Kong authorities said a finance employee received a message in January 2024 from someone impersonating the chief financial officer of the employee's UK-based head office.
The employee was invited into what appeared to be a group video conference involving senior colleagues. Police later said the conference had been fabricated using publicly available video clips and voices of the people being impersonated.
The employee ultimately authorised transfers worth about HK$200 million to five Hong Kong bank accounts. British engineering company Arup later confirmed that it was the company affected by the fraud.
The case matters because the deception did not depend on one strange audio message from an unknown number. It reproduced something considerably more reassuring: the social environment of a workplace.
Familiar faces appeared together. The request seemed to come from inside an existing hierarchy. What looked like confirmation from several colleagues was actually part of the same deception.
Hong Kong police said the conference was pre-recorded rather than a genuine live interaction. That detail is important. A fabricated video does not necessarily need to respond naturally for an extended conversation if the victim has already been given a reason why the meeting is brief, confidential or unusually structured.
The dangerous threshold is not perfect imitation. It is believable imitation under pressure.
Voices are becoming identity material
Voice cloning demonstrates the broader problem particularly clearly.
People recognise one another through far more than words. Cadence, hesitation, pronunciation, breath and tone all contribute to the feeling that a voice belongs to somebody familiar.
Synthetic systems can reproduce enough of those characteristics to create persuasive audio. The listener does not need to believe the recording would survive laboratory analysis. They need to believe it for the few seconds in which a decision is being requested.
Public life also leaves enormous quantities of voice material behind. Interviews, podcasts, social-media videos, livestreams and conference appearances can all place recognisable speech online.
The FBI has repeatedly warned that criminals can combine impersonation with pressure tactics and has advised people to verify unexpected requests through previously established contact information rather than trusting a new number or account simply because the voice sounds right.
The numbers are no longer hypothetical
The FBI's 2025 Internet Crime Report, released in 2026, included a dedicated section on artificial intelligence in cybercrime.
AI-related complaints recorded by the FBI's Internet Crime Complaint Center for 2025.
Adjusted losses associated with those complaints.
Reported 2025 business-email-compromise losses in cases with an AI element.
Those figures should not be read as a measurement of every deepfake scam. AI can appear in many forms of fraud, and not every deepfake incident is reported or identified as such.
They do show that AI-assisted deception has moved beyond demonstrations and speculative security warnings.
Politics discovered the same weakness
A synthetic recording does not need to steal money directly to cause damage.
Before New Hampshire's 2024 presidential primary, voters received robocalls using an AI-generated imitation of then-President Joe Biden's voice. The calls falsely encouraged recipients not to vote in the primary.
In September 2024, the US Federal Communications Commission imposed a $6 million fine on political consultant Steve Kramer over the illegal robocall campaign, which also used caller-ID spoofing.
The example exposed a different asymmetry from financial fraud.
A deceptive political recording can be distributed almost instantly. Verification usually takes longer. Reporters may need to locate the original recording, contact the person depicted, establish who first distributed the material and compare it with independent evidence.
The fake therefore receives a head start.
Deepfakes create a second problem: denying what is real
Synthetic media does not only make false evidence easier to manufacture.
It can also make genuine evidence easier to dismiss.
A public figure confronted with authentic audio or video now has a ready-made explanation available: artificial intelligence made it.
Researchers have sometimes described this broader phenomenon as a liar's dividend. The existence of realistic fakes creates doubt that can be exploited even when the disputed recording is authentic.
That makes indiscriminate skepticism dangerous too.
If the response to deepfakes becomes "nothing can be trusted," fabricated media has damaged the information environment even when nobody believes the original fake.
Detection is useful, but it is not a magic verdict
The obvious technical response is automatic detection.
If artificial intelligence can generate synthetic media, another system might identify the traces left behind.
Media-forensics research is active, including work evaluated through programmes such as the US National Institute of Standards and Technology's Open Media Forensics Challenge.
But a detector should not be treated as an oracle.
Generation methods change. Files are compressed and resized. Social networks re-encode uploads. Images are cropped. Videos are screen-recorded. Audio is transmitted through telephone networks or messaging applications.
Every transformation can change the signals available to forensic software.
A percentage displayed by a detection service may therefore be one piece of evidence, not the end of the investigation.
Sometimes the best forensic question is boring
Where did the file come from?
That question can be more valuable than searching a face for visual glitches.
Viral media often travels through reposts that strip away its original caption, timestamp and relationship to the person who first published it.
Finding an earlier version can reveal that a dramatic clip was shortened. It may reveal a different description of the event. It may show that the supposed original account never posted the material in the first place.
The same applies to screenshots.
A screenshot of a social-media post is an image claiming that a post existed. If the alleged post cannot be found through an archive, direct source or other independent record, the screenshot itself should not automatically be treated as proof.
Provenance is becoming part of the answer
One response to synthetic media is to move some of the verification process closer to the moment a file is created or edited.
The Coalition for Content Provenance and Authenticity, or C2PA, develops an open technical standard for attaching tamper-evident information about the origin and history of digital media.
Its Content Credentials system can record claims about how an asset was created, what tools altered it and which earlier materials were used.
The standard continued to evolve in 2026. C2PA released version 2.4 of its technical specification in April and later published additional implementation guidance on identifying synthetic and non-synthetic material.
Provenance is fundamentally different from a detector trying to guess whether pixels look synthetic. It attempts to provide a chain of information about what happened to the asset.
But provenance also has limits.
Not every camera or editing tool supports the same system. Credentials can be absent from legitimate older material. A screen recording may preserve the visible content of an authentic source while losing the original provenance information.
Missing credentials cannot therefore be treated as proof of fabrication.
Europe has moved from guidance to legal obligations
The regulatory environment is changing as well.
Article 50 of the European Union's AI Act began applying on 2 August 2026.
Among its transparency requirements, deployers of AI systems that generate or manipulate image, audio or video constituting a deepfake must disclose that the content has been artificially generated or manipulated, subject to specified exceptions.
The rules also require providers of relevant generative systems to support machine-readable marking of synthetic or manipulated output, although transitional timing applies to some systems placed on the market before August 2026.
The principle behind the law is straightforward: people should not have to discover by accident that the person, voice or event they are seeing was synthetically generated.
Britain has focused particularly on intimate deepfakes
The harm from synthetic media is not confined to fraud or politics.
Non-consensual sexual deepfakes can create images appearing to show a real person naked or involved in sexual activity that never occurred.
In the UK, provisions brought into force on 6 February 2026 criminalised creating, or requesting the creation of, purported intimate images of adults without consent or a reasonable belief in consent.
Further provisions under the Crime and Policing Act 2026 came into force in June, including offences directed at making or supplying tools designed for generating purported intimate images.
The legal response reflects something technical discussions sometimes miss: a deepfake does not need to convince the entire internet to damage the person depicted.
The harm may come from humiliation, harassment, professional consequences or the simple fact that a victim is forced to prove that an image of their own body is fictional.
Real people, synthetic identities
Deepfakes also blur into a broader category of synthetic identity.
An account can combine generated photographs, synthetic video, cloned voices and machine-written messages to create the appearance of a person who does not exist, or to impersonate somebody who does.
Not every virtual identity is deceptive. Fictional characters, games, entertainment and openly artificial personalities can use the same technology without pretending to be documentary reality.
The problem begins when the artificial nature of the identity is concealed in a context where the other person reasonably believes they are dealing with a real individual.
What journalists now have to verify
Synthetic media increases the cost of publishing quickly.
A dramatic recording can appear during a breaking event and gather millions of views before a newsroom has identified who created it.
Popularity does not solve that uncertainty.
A million views do not make a clip authentic. They make the consequences of publishing the wrong conclusion larger.
Verification may involve tracing the earliest available version, comparing footage from the same event, checking whether geography and weather match, examining metadata, contacting the person supposedly depicted and finding witnesses who can independently confirm what happened.
None of those checks works in every case.
The underlying principle does.
Evidence has to be reconstructed rather than assumed from appearance.
Businesses need procedures that survive convincing impersonation
The Hong Kong case points toward a simple defensive lesson.
An organisation should not design a high-value approval process around the assumption that recognising somebody's face or voice proves their identity.
An unusual financial instruction can be verified through a separate, previously established channel. Sensitive changes can require independent approval. Staff can be encouraged to stop when urgency is being used to bypass an ordinary procedure.
The same logic applies outside companies.
Families can agree that an emergency request involving money will be checked by calling a known number rather than replying only to the account that initiated the request.
Practical verification
What ordinary people can realistically do
- Slow down when a message creates artificial urgency.
- For money, passwords or sensitive information, confirm the request through another known channel.
- Do not assume a familiar voice, face or caller identity is enough on its own.
- Look for the original source of dramatic video rather than relying only on reposts.
- Treat screenshots without a verifiable underlying source cautiously.
- If a supposed contact suddenly uses a new number or account, verify the change independently.
The wrong response is to trust nothing
The deepfake problem can easily produce the opposite mistake from blind trust.
If people conclude that every photograph could be artificial, every recording could be cloned and every video could be fabricated, authentic evidence becomes easier to ignore.
Photographs still document real events.
Audio still records real speech.
Video still provides powerful evidence.
What is changing is the amount of work required before appearance alone can settle an important question.
Who created the file?
Where did it first appear?
Can the people involved confirm it?
Does independent evidence support what it seems to show?
Seeing and hearing still matter.
They just no longer end the investigation.
Reporting note
Sources used for this article
Factual reporting for this article draws on public material from the Hong Kong Government and Hong Kong Police, the US Federal Communications Commission, the FBI and its Internet Crime Complaint Center, the UK Government and UK legislation, the European Commission and the European Union AI Act, the National Institute of Standards and Technology, and the Coalition for Content Provenance and Authenticity.
If you believe this article contains a factual error, visit our corrections page .
