OpenAI says it has disrupted a Cambodia-based criminal operation that used ChatGPT to support romance scams, fake investments, online gambling schemes, and law enforcement impersonation. The company banned a coordinated network of accounts that it believes originated in Cambodia and operated in or around Poipet, a city repeatedly linked to scam compounds and human trafficking.
According to OpenAI’s report, the scammers used ChatGPT to create fake online personalities, translate conversations, draft messages to potential victims, and produce promotional material. Some users also generated forged passports, legal notices, stock purchase confirmations, and fake cryptocurrency trading interfaces.
The operation apparently followed a familiar pattern. Scammers contacted targets through platforms such as WhatsApp and Telegram, built trust through romantic conversations or promises of guaranteed investment returns, and eventually demanded deposits, activation fees, or fictitious fines.
OpenAI says the investigation began with information provided by WhatsApp. It later shared additional threat indicators with industry partners and authorities, although the company has not revealed how many accounts it banned or identified the people behind the operation.
Perhaps the darkest part of the report involves signs of human trafficking and forced labor. Some ChatGPT users created advertisements promising flights, housing, meals, visas, and work permits for so-called chatter jobs in Poipet, while other conversations discussed employee debts, salary deductions, disciplinary fines, visa overstays, detention, and escape attempts.
In other words, at least some of the people sending scam messages may have been victims themselves. Organized crime groups in Southeast Asia have been accused of luring workers with legitimate-looking job offers before forcing them to participate in online fraud.
OpenAI says the network may have contacted hundreds of targets, with internal conversations mentioning individual losses of thousands of dollars. However, the company admits it could not independently verify those claims, and the full financial impact remains unknown.
It is good that OpenAI removed the accounts, but the report also demonstrates how useful generative AI can be to organized criminals. ChatGPT did not invent romance scams or fake investment platforms, of course, but it apparently made it easier for one operation to create convincing identities, communicate across languages, and run several types of fraud at once.
Banning the accounts may disrupt this particular network, but it probably will not end the operation. The scammers can create new accounts, switch AI services, or use open models that have no central company capable of shutting them down.
Support independent tech journalism
NERDS.xyz is independently owned and operated. If you enjoy my coverage of Linux, AI, hardware, cybersecurity, and tech culture, consider supporting the site on Ko-fi.
Support NERDS.xyz