Explainer · Adapt Better
How AI raises the social engineering risk
AI does not invent new human weaknesses. It industrialises the old ones, making the same manipulation cheaper, faster and far more convincing.
How AI makes social engineering easier
AI has not changed the levers attackers pull. Urgency, authority and helpfulness are still the target. What it has changed is the cost and the polish: convincing fakes at scale, with the tells that used to trigger doubt removed.
What AI-powered social engineering looks like
A voice or face on a call that looks exactly like a colleague. Phishing with no clumsy grammar to give it away. Chatbots and assistants talked into ignoring their own rules. And the AI tools themselves becoming something worth attacking.
Why these attacks work
Social engineering works by loading the moment, pressure, a senior name, a reason to help, so the shortcut runs before judgement catches up. AI removes the friction that used to interrupt that: the wrong face, the odd phrasing, the delay.
It turns out a model has its own version of the same weakness too, its guardrails can be talked around much as a person’s caution can. And the AI supply chain, the models and tools teams now trust, is a fresh surface to poison.
How to defend against it
The defence changes less than it looks. The behavioural move still holds: slow the urgent request, verify on a second channel, make it normal to check.
What is added is treating AI as attack surface too, provenance for the tools and models you adopt, and basic secrets hygiene, so a convincing fake meets a habit that does not bend under pressure.
The evidence
These are recent, documented cases, cited to show the pattern rather than to single anyone out; every platform in this space faces the same pressure. In February 2024 a finance worker at the engineering firm Arup was deceived into transferring about US$25 million after a video call in which every other participant was an AI-generated deepfake of senior colleagues. In August 2024 researchers showed Meta’s open-source prompt-injection filter, PromptGuard, could be walked past simply by spacing out the letters of a malicious instruction, dropping its detection rate from near 100% to under 1%; Meta acknowledged the issue. And in 2024 the AI model hub Hugging Face disclosed unauthorised access that may have exposed a subset of stored secrets, while security firms separately found models uploaded to the hub that run hidden code when loaded.
- CNN, 4 February 2024. Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’. www.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk
- SC Media, 2024. Meta’s PromptGuard model bypassed by simple jailbreak, researchers say. www.scworld.com/news/metas-promptguard-model-bypassed-by-simple-jailbreak-researchers-say
- The Hacker News, June 2024. AI Company Hugging Face Detects Unauthorized Access to Its Spaces Platform. thehackernews.com/2024/06/ai-company-hugging-face-notifies-users.html
Get your team ready for AI-powered scams
Think like an attacker, then practise the pause, so a convincing fake meets a habit that holds.
The Signal
One behavioural idea worth keeping, now and then. No noise, just signal. Leave your email and we will send the good stuff.
Filed under Adapt Better. By MindHug. Last updated 5 November 2026.