OpenAI’s latest update to ChatGPT, which enables native image generation through its GPT-4 model, has taken the internet by storm. Since its release last week, users have created over 700 million images using the new feature. While many use it to design artistic content—especially Studio Ghibli-style portraits—the powerful tool is also drawing criticism over potential misuse.
AI Tool Generates Fake Aadhaar and PAN Cards
There is increasing concern since social media users have started sharing fake Aadhaar cards created through ChatGPT. The photos mimic actual government-issued documents with breathtaking accuracy. In certain cases, even fake Aadhaar cards of OpenAI CEO Sam Altman and Tesla CEO Elon Musk were shared by users, along with Aadhaar numbers and QR codes.
Ok, so ChatGPT can create Aadhaar images. Thats not the interesting thing. The interesting thing is, where did it get the Aadhar photo data for training? pic.twitter.com/kb6lvuD04E
— nutanc (@nutanc) April 3, 2025
As a test of the tool’s ability, an effort was made to create an Aadhaar card-like photo with ChatGPT. The result was uncannily similar to an actual Aadhaar card—while the facial features were still not consistent, it suggested that the AI was not creating perfect duplicates but perilously close substitutes.
ChatGPT is generating fake Aadhaar and PAN cards instantly, which is a serious security risk.
This is why AI should be regulated to a certain extent.@sama @OpenAI pic.twitter.com/4bsKWEkJGr
— Yaswanth Sai Palaghat (@yaswanthtweet) April 4, 2025
It does not stop there. ChatGPT’s in-built image-generation ability has also been exploited to produce counterfeit Permanent Account Number (PAN) cards, another essential identification document issued by India’s Income Tax Department. Online users noted how the AI-generated images could mimic the layout and contents of a PAN card with high precision.
OpenAI Acknowledges New Risks With GPT-4
New ability means greater responsibility. ChatGPT’s image generation is no longer dependent on separate models, such as DALL·E 3, but natively fueled by the GPT-4o model. It thus creates more accurate and realistic content from user inputs.
OpenAI admitted greater danger in its system card for GPT-4o, mentioning, Unlike DALL·E, which operates as a diffusion model, 4o image generation is an autoregressive model natively embedded within ChatGPT. This fundamental difference introduces several new capabilities that are distinct from previous generative models, and that pose new risks.”
The firm continued that such new capabilities “have the potential to create risks across a number of areas, in ways that previous models could not.”
OpenAI Implements Content Restrictions
To prevent abuse, OpenAI has put tough protections in place in ChatGPT’s image generator. They include limits on producing:
- Photorealistic pictures of children (including young public figures)
- Adult content
- Abusive, violent, or hateful images
Nonetheless, increasing misuse cases are bringing back into the limelight the debate about AI regulation, especially in identity fraud. While AI tools improve and become easily available, the experts caution that active regulation and technical protection measures are more vital than ever now.
