The Blind Spot: What a 2023-Trained AI Image Detector Sees in a 2026 Image

You are watching the wrong signal. A detector older than the generator in front of it usually does not return "uncertain" — it returns real, at a confident percentage, and it returns the same number every time you resubmit. That is not caution. It is a structural gap. A real-versus-fake classifier has no third class for "I have never seen this", so anything outside its training distribution falls to the negative side. Ojha et al. (2023, arXiv:2302.10174) describe the mechanism plainly: the real class becomes "a sink class holding anything that is not fake, including generated images from models not accessible during training". The sections below are how to recognise that verdict in the wild, and how to test for it in ten minutes without a lab.

What "trained in 2023" actually pinned down

A detector of that generation did not learn the idea of synthesis. It learned the manufacturing residue of a specific stack, and that stack had three visible parts.

The autoencoder. Latent diffusion compresses an image into a smaller representation and decodes it back, and the decoder is a convolutional network with a fixed upsampling pattern (Rombach et al., 2022, arXiv:2112.10752). That pattern leaves a periodic residue in the output. It is faint, it is everywhere, and it is exactly the kind of regularity a convolutional classifier is good at finding.

The denoiser. The U-Net that removes noise at each step imprints its own texture onto mid-frequency detail. Two models trained on the same data with different schedules do not produce the same texture.

The crop. Training crops are small. Whatever the model learned, it learned at that size, which means the tool must resize or tile your file before inference — and that step removes or rescales part of the residue itself.

Now change the stack. Rectified flow, as used in the transformer-based generators that arrived from 2024 onward, "connects data and noise in a straight line" rather than inverting a diffusion path, and pairs it with a transformer backbone instead of a convolutional U-Net (Esser et al., 2024, arXiv:2403.03206). Different trajectory geometry, different decoder, different backbone. The residue a 2023 detector was fitted to is not necessarily gone — but it is no longer the residue of the process it was fitted to.

The practical consequence: the detector is not wrong about the file, it is answering a question about a process that no longer runs. No vendor publishes whether their model was retrained after a given release, so you cannot tell from the landing page which process a tool is currently pointing at.

Why the failure is confident instead of cautious

Most people expect a stale model to hesitate. It does the opposite, and the reason is in how the boundary is drawn.

A binary classifier is trained to separate one thing from everything else. The "fake" side is learned actively, because that is the class the loss pushes on. The "real" side is learned passively, as the complement. Push an input far outside the fake cluster and it lands deep in the complement — not near the boundary where a model might waver, but in the middle of the region that has always been labelled real. Distance from the fake cluster reads as confidence, so an unfamiliar generator produces a stronger "real" than an ambiguous photo does. This is the inversion that trips people up: the more alien the image, the more certain the verdict.

This was measurable in 2023. The same paper that named the sink class found that dropping the learned classifier in favour of nearest-neighbour search in a frozen pretrained feature space "improves upon the SoTA by +15.07 mAP and +25.90% acc when tested on unseen diffusion and autoregressive models" (Ojha et al., 2023, arXiv:2302.10174). Read that as a statement about the state of the art at the time: the gap between "works on generators it saw" and "works on generators it did not" was large enough that removing the classifier entirely beat it.

Four shapes the failure takes

These are the observable forms, not four different bugs. The same gap produces all of them depending on what you feed it.

1. Refusal on the obvious. The image a person calls instantly synthetic — six fingers, a staircase that goes nowhere, text that reads as letter-shaped noise — comes back real. This looks like a malfunction and is not one. The human is reading semantics; the model is reading residue. A 2023 detector has no opinion about anatomy, and it never claimed to.

2. The stable wrong number. Run the same file twice and you get the same label and essentially the same percentage. People take repeatability as reliability. It is not, here. An ambiguous input wobbles because two clusters compete for it; an out-of-distribution input does not wobble, because nothing competes. Consistency tells you the model is not sampling randomly. It says nothing about whether its region of certainty is the right region.

3. Sensitivity to crop and resize. A tool that resizes before inference is not answering a question about your image, it is answering one about its own preprocessing. Feed the full frame, then feed a tight crop of a flat area like sky or a wall, then a crop of a detailed area. If the three disagree, the number is tracking scale, not content.

4. The flip on re-encode. Save the file once as JPEG and run it again. Nothing about the scene changed; the container did. A verdict that moves is a verdict about the container. A verdict that holds is not thereby correct — it only means this particular artefact is not what the model was reading.

None of these four proves a blind spot. Together they narrow it down enough to stop treating the number as evidence.

The ten-minute control test

The mistake almost everyone makes is testing the suspect file alone. One score, no comparison, no way to attribute the result. Detection error has to be bracketed with controls, the way any measurement is.

Step 1 — Build three files, not one. The suspect image, exported at native resolution as PNG. A file you know is synthetic and know is old: something you generated with a 2022 or 2023 model and still have. And a camera original with its EXIF intact. These are your positive control, your reference, and your negative control.

Step 2 — Freeze them before you touch a browser. No screenshots, no chat-app forwarding, no cloud re-compression. Record a SHA-256 for each so you can prove later that the same bytes went in.

Step 3 — Run each file twice and write down the raw label and number. Not "it said AI" — the exact output string and percentage. Repeatability is part of the reading.

Step 4 — Read the pattern, not the score.

Step 5 — Only if you got row one, escalate. Crop test at three scales, then the single JPEG re-encode. If the verdict follows the crop and flips on the re-save, you have a detector reading texture rather than content, and you can say so with a specific reason instead of a feeling.

The whole thing costs three uploads and a text file. It also produces something a vendor's accuracy figure cannot: a result specific to your file and your question, with a control next to it.

What a clean verdict actually licenses you to say

Almost nothing, and less than it used to. "Real" is not a finding; it is the absence of a detected signal. The strength of that absence depends entirely on the detector's coverage, and for a 2026 file against a model trained before it, coverage is unknown. An unknown denominator makes the absence uninformative.

This is why a low score should never be quoted as evidence of authenticity in a dispute. It is not a weaker form of proof, it is the wrong kind of statement — the same category error as reading a metal detector's silence as proof that the ground is empty. The percentage page covers why the number itself cannot carry that weight; if you are already in a dispute, the false-positive walkthrough is the one that maps to a reviewer's decision.

The one check that does not age the same way

A provenance signal is inserted at generation and verified against a key, so it does not depend on the verifier having seen that generator during training. That is a different mechanism, not a better model. C2PA Content Credentials and Google's SynthID both work this way, which is why a hit on a 2026 file is meaningful in a way a pattern score is not.

The asymmetry is worth stating exactly: a provenance hit is evidence that a participating tool handled the file. A miss is not evidence of anything, because most images in the world carry no credential at all. Google's own page for its SynthID tool says it "is not a general AI detector" and only speaks for content from companies that adopted the technology. The SynthID walkthrough covers how to read a hit and a miss without overclaiming either.

This is a known, measured problem, not a theory

Two benchmarks exist for the reason this page exists. GenImage was built with "over one million pairs of AI-generated fake images and collected real images" and defines a cross-generator task — train on one generator, test on the others — alongside a degraded-image task for low-resolution, blurred and compressed files (Zhu et al., 2023, arXiv:2306.08571). AI-GenBench goes further and orders training data by generator release date, so that a detector is always evaluated on models newer than the ones it learned from (Pellegrini et al., 2025, arXiv:2504.20865). Both were built because a single accuracy figure measured on a fixed snapshot is not a durable claim about tomorrow's images.

If you are choosing a tool rather than defending a file, that is the question to ask it: not "how accurate were you", but "when was the last generator you were trained against released". A tool that cannot answer has an unknown expiry date, and the comparison of the main detectors should be read with that in mind.

FAQ

The detector said 2% AI. Isn't that almost certainly a real photo? No. A low number is the absence of a detected signal, and its meaning depends on whether the model ever learned the signal for that generator. For a post-2024 generator shown to a 2023-trained model, the number is closer to "no data" than to "probably real".

Does this mean every clean verdict is worthless? No. If the file came from a generator the model was trained on, a clean verdict is a genuine miss by the tool and worth investigating. The test above is how you tell the two apart before you rely on either.

Do paid detectors avoid this? Not automatically. A subscription buys throughput, an API and sometimes a newer model, but no vendor in the pages reviewed for this article publishes a training cut-off date. Ask for that date; it is the only number that predicts this failure.

Why does it say real even on images with obvious errors? Because it is not looking at the errors. A pattern detector reads low-level statistics, and anatomy, physics and text are semantic properties it was never trained on. That is also why "it looks obviously fake to me" and "the detector says real" can both be true.

How often should a detector be retrained to stay useful? There is no published interval, and that is part of the problem. AI-GenBench exists precisely to evaluate detectors against generation-ordered data instead of a frozen set (Pellegrini et al., 2025, arXiv:2504.20865). Treat any tool's accuracy claim as scoped to the generators it was tested on, and assume the scope is older than the claim sounds.

Sources

• Ojha, Li, Lee, "Towards Universal Fake Image Detectors that Generalize Across Generative Models", 2023: arxiv.org/abs/2302.10174
• Rombach et al., "High-Resolution Image Synthesis with Latent Diffusion Models", 2022: arxiv.org/abs/2112.10752
• Esser et al., "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis", 2024: arxiv.org/abs/2403.03206
• Zhu et al., "GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image", 2023: arxiv.org/abs/2306.08571
• Pellegrini et al., "AI-GenBench: A New Ongoing Benchmark for AI-Generated Image Detection", 2025: arxiv.org/abs/2504.20865
• Google DeepMind, SynthID overview: deepmind.google/models/synthid/