ChatGPT’s Image Generator Leveraged to Create Fake Passport

Researchers have recently discovered that OpenAI’s ChatGPT image generation feature can be manipulated to create convincing fake passports with minimal effort.

This capability, introduced on March 25, 2025, for ChatGPT-4o and ChatGPT-4o mini models and made freely available to all users by March 31, has quickly been exploited beyond its intended creative purposes.

The revelation represents a significant shift in the accessibility of document forgery tools, transforming what once required specialized skills and underground connections into a simple process requiring only a few carefully crafted prompts.

Unlike previous forgery methods that demanded technical expertise and specialized software like Adobe Photoshop, these AI-generated documents can be created by individuals with no prior experience in cybercrime.

Cato Networks analysts identified this vulnerability through systematic testing of the image generation capabilities.

According to their research, creating fake identification documents takes mere minutes and doesn’t require any complex “jailbreaking” techniques to circumvent ChatGPT’s safety measures.

This democratization of forgery tools enables what security experts call “zero-knowledge threat actors” – individuals who can execute sophisticated fraud despite having no background in cybercrime.

The bypass technique exploited by researchers involves a simple yet effective social engineering approach to circumvent ChatGPT’s content policy restrictions.

When directly asked to modify a passport image, ChatGPT appropriately refuses due to privacy and legal concerns.

However, when the same request is reframed as modifying a “business card designed to look like a passport,” the system readily complies, complete with changed names and swapped photos.

The researchers were able to request changes to the passport by claiming it was actually a business card created for someone who “used to work at customs and border protection.”

This simple misdirection was sufficient to convince the AI to alter the document, maintaining authentic-looking stamps, fonts, and formatting—essential elements for creating convincing forgeries.

This development raises serious concerns about identity verification processes in an era where increasingly realistic fake documents can be created without specialized knowledge.

The implications extend beyond cybersecurity into domains such as banking, border control, and government services that rely on document verification for security.

Equip your team with real-time threat analysis With ANY.RUN’s interactive cloud sandbox -> Try 14-day Free Trial

The post ChatGPT’s Image Generator Leveraged to Create Fake Passport appeared first on Cyber Security News.