Kush_EXE

Can you trust ChatGPT to tell you if a photo is AI?

· Runs 2:44

ChatGPT made this picture of a man at a mailbox. Then, when researchers at the Brennan Center for Justice asked six chatbots whether it was AI-made, ChatGPT studied the shadows, the hands and the street sign and called it very likely genuine. Grok said there was a real mailbox on that street in San Mateo, California. Claude guessed stock photo, and Meta AI called it real. Only Gemini read the hidden OpenAI label the file carried, the kind of label California's AI Transparency Act has required since August 2. A mid-September rerun, after the law took effect, found nothing had improved.

We also go through their bigger August study, where ChatGPT and Grok didn't flag a single image their own tools had made, plus why a screenshot can wipe that hidden label and why a missing label proves nothing. The takeaway: a chatbot saying "looks real" is a guess, not a fact-check. Check who else is reporting the image: fact-checkers like PolitiFact or AP Fact Check, and for anything about voting, your county election office.

Watch on YouTube Subscribe

In this video

  1. ChatGPT called its own fake real
  2. Six chatbots, one photo
  3. The hidden label only Gemini read
  4. The new law and the September rerun
  5. The bigger study: 14 AI images
  6. Why labels won't save you either
  7. Pixels lie: how to actually check a photo

ChatGPT called its own fake real 0:00

You'd think ChatGPT knows its own fakes. It called this very likely real. Say you're scrolling at night. A photo stops you: a man at a mailbox. Something feels off, so you ask a chatbot: is this real?

Six chatbots, one photo 0:14

Researchers at the Brennan Center for Justice did exactly that, with six chatbots and this exact picture. ChatGPT studied the shadows, the hands and the street sign. Its verdict: very likely genuine. Grok went further. It said there's a real mailbox on that street in San Mateo, California, and the houses and palm trees match. Claude saw no sign of fakery and guessed stock photo. Meta AI called it real, and Perplexity said it didn't look obviously AI.

The hidden label only Gemini read 0:42

And here's the part that stings: the file carried a hidden label from OpenAI, saying where it came from. Only Gemini did the boring, correct thing: it read that label and named OpenAI as the maker. None of the others even mentioned it. Not even ChatGPT, the one that made the picture.

The new law and the September rerun 1:00

That label isn't a nice extra. California's AI Transparency Act took effect August second, and it requires AI images to carry one. That mailbox test was the mid-September rerun, after the law took effect. Nothing had improved, and Gemini was still the only one checking.

The bigger study: 14 AI images 1:16

And this wasn't one bad day. In their earlier, bigger study of 14 AI-made images, ChatGPT, Grok, Claude and Perplexity each caught four or fewer. Gemini and Meta AI caught most. But ChatGPT and Grok didn't flag a single image their own tools had made. To be fair, after that study OpenAI released a separate checking tool, for its own images only. Retested, ChatGPT caught some of its own images. Not all. And these were a small set of test images from one research team, not every photo out there.

Why labels won't save you either 1:48

But don't crown Gemini either. It reads labels, and pictures with no readable label, it couldn't identify. And that label is fragile. OpenAI says a screenshot or a file conversion can wipe that metadata. So the photo you saw, screenshotted and reshared, may have no label left. And a missing label proves nothing, because most real photos don't have one either.

Pixels lie: how to actually check a photo 2:09

Label or no label, most of these chatbots did what you'd do: squint at pixels. And pixels lie. As the Brennan Center's Abdiaziz Ahmed put it, "a chatbot's answer is not verification." So don't zoom in. Zoom out: if that really happened, would real news outlets be covering it? Check fact-checkers like PolitiFact or AP Fact Check, and for anything about voting, your county election office. Before you hit share, remember one thing. A chatbot saying "looks real" is a guess, not a fact-check.

What the sources say

  • The Brennan Center for Justice tested whether six chatbots (ChatGPT, Claude, Gemini, Grok, Meta AI and Perplexity) could tell when an image was generated by AI. In an analysis by Abdiaziz Ahmed published on October 5, 2026, it reported that most could not, even with images their own products had made. Brennan Center for Justice
  • The Brennan Center for Justice reported on October 5, 2026 that, in its test of six chatbots, ChatGPT was shown an image of a mailbox that it had generated and that carried OpenAI's metadata. It examined the consistency of the shadows, the hands and the street sign and concluded the photo was very likely genuine. Brennan Center for Justice
  • In a Brennan Center for Justice test, Grok called a ChatGPT-generated mailbox image a real photo and said a real postal collection box stood on the street shown, in San Mateo, California. Claude found no evidence of fakery, Meta AI called it a real photo, and Perplexity said it did not look obviously AI-generated. Brennan Center for Justice
  • In a Brennan Center for Justice test of six chatbots on two ChatGPT-made images carrying OpenAI's metadata, only Gemini detected the metadata, naming OpenAI as the creator; ChatGPT did not mention metadata. A Brennan Center report says Gemini flagged images carrying provenance data but could not identify the rest. Chatbots Still Bad at Spotting AI-Generated Images Despite New Law, Does AI Fight or Fuel Election Disinformation?
  • The Brennan Center for Justice reported that ChatGPT, Grok and Perplexity correctly identified a ChatGPT-made test image of FBI agents and voter-registration intimidation as AI-generated, but only because the scene looked staged and searches found no real source. Claude and Meta AI called the image real. Brennan Center for Justice
  • The Brennan Center for Justice reran its tests of whether chatbots can spot AI-generated images in mid-September 2026, after California's AI Transparency Act took effect, which its report dates to August 2, 2026. Performance had not improved, and Gemini was still the only chatbot that checked the images' metadata. Chatbots Still Bad at Spotting AI-Generated Images Despite New Law, Does AI Fight or Fuel Election Disinformation?
  • In a Brennan Center study of 14 AI-generated images, published on August 11, 2026, Gemini and Meta AI correctly identified most as AI, while ChatGPT, Grok, Claude and Perplexity identified 4 or fewer. ChatGPT and Grok did not flag any of the images their own tools had made. Brennan Center for Justice
  • The Brennan Center reported in August 2026 that after OpenAI released a public verification tool, ChatGPT, retested, detected some of its own images but not all. OpenAI says its verification tool only detects content from ChatGPT, the OpenAI API or Codex, and that finding no signal does not rule out AI generation. Brennan Center for Justice, OpenAI
  • OpenAI says Content Credentials, the C2PA label on its images, can be lost through everyday actions such as taking a screenshot or converting a file. Critics cited in Tech Policy Press note that a missing label does not necessarily mean an image is AI-generated, since most authentic photos carry no label either. OpenAI Help Center, Tech Policy Press
  • In an October 5, 2026 analysis of how well chatbots spot AI-generated images, the Brennan Center's Abdiaziz Ahmed wrote that "a chatbot's answer is not verification." The analysis names official election office websites and fact-checkers such as PolitiFact and AP Fact Check as better starting points. Brennan Center for Justice

Sources

  1. Chatbots Still Bad at Spotting AI-Generated Images Despite New Law Brennan Center for Justice
  2. Does AI Fight or Fuel Election Disinformation? Brennan Center for Justice
  3. LLMs Are Still Making Election Deepfakes Just Days Before the US Midterms Tech Policy Press
  4. Verify OpenAI-generated content OpenAI
  5. Provenance signals in OpenAI-generated content OpenAI Help Center
  6. California Revamps AI Transparency Requirements Morrison Foerster

More in AI news explained