Why Every AI Model Comparison Is Out of Date (And How to Check)

In August 2026 I audited a directory of 251 AI tools against the vendors' own documentation. Of the fourteen flagship model entries, thirteen were wrong.

Not wrong at the margins. Wrong by whole generations.

I should be straight about whose directory it was: ours. This site's AI tools catalog had been seeded carefully, rated across capabilities, wired into a recommender — and then left to rot for a few months, which turns out to be all it takes. Every competing directory and best AI models listicle I checked while writing this is in the same state or worse. The difference is that I went and looked.

Here is what a few months of drift produced:

Listed as current	Actually current (vendor docs, 15 Aug 2026)
GPT-5	GPT-5.6 Sol, with Terra and Luna variants
Claude Opus 4.8, Sonnet 4.6	Opus 5, Sonnet 5 — both older ones now filed under Legacy models
Grok 4	grok-4.6, alongside grok-4.5 and grok-4.3
DeepSeek-R1, DeepSeek-V3	deepseek-v4-pro, deepseek-v4-flash
Gemini 3	Gemini 3.7 Flash leading the stable line
Mistral Large	Mistral Large 3
Kimi	kimi-k3
Qwen3	qwen3.7-max
Command A	Command A+
Nano Banana	Nano Banana Pro, Nano Banana 2
GPT Image	gpt-image-2

One entry survived: Llama 4, which Meta still lists as its current generation.

Why this happens to everyone

It would be comfortable to call this laziness. It isn't, quite. Three structural things make model content decay faster than almost any other kind.

The publishing model is wrong for the subject. A comparison article is written once, published, and then optimised — internal links, meta description, maybe a refreshed date stamp. Nothing in that workflow re-reads the vendor's docs. The page accrues authority while its contents quietly become false, and the ranking signal and the accuracy signal move in opposite directions.

Version numbers are not linear and not comparable. Anthropic went 4.5 → 4.6 → 4.7 → 4.8 → 5. Google runs 3.1, 3.5, 3.6 and 3.7 concurrently, at different stability levels. xAI has 4.3, 4.5 and 4.6 live at once with different context windows. There is no rule that lets you infer the current version from the one you wrote down last time, so the only way to know is to look.

Products get renamed and acquired without announcement. This is the one that catches even diligent writers, because the product still works and the name still resolves. More on it below.

Three traps in the current lineup

These aren't pedantry — each one will cost you money or a failed deployment if you get it backwards.

The highest version number is not the most production-ready. Google's stable Gemini 3 line is led by Gemini 3.7 Flash. Gemini 3.1 Pro — lower number, higher tier — is in preview. Pick by the number and you ship a preview model to production; pick by the tier name and you may skip the stable one entirely.

The newest model is not the largest-context model. xAI's grok-4.6 carries a 500k-token window and its docs call it the most intelligent and fastest model we've built. grok-4.3, an older sibling, carries 1M. If your workload is defined by document size rather than reasoning difficulty, the newest release is the wrong choice — see what a context window actually is for why bigger isn't automatically better either.

A successor can be smaller than what it replaces. Cohere's Command A+ (command-a-plus-05-2026) has a 128k context window with 64k max output. The older Command A has 256k of context but only 8k of output. Neither dominates: one reads long, the other writes long. A comparison table with a single context window column cannot express that, which is exactly why most of them mislead.

And a fourth, quieter one: legacy does not mean gone. Anthropic files Claude Opus 4.8 under Legacy models, but it still serves on the API at $5 per million input tokens and $25 per million output, with a 1M context window. Legacy means don't start here, not this stopped working. Plenty of production systems are running legacy models perfectly well and have no urgent reason to move.

The rebrand problem, and the redirect test

While checking tools rather than models, I found three products that had changed hands or changed names with no visible signal on the pages that list them:

NotebookLM is branded Gemini Notebook on Google's own support pages, and notebooklm.google.com now 301-redirects to notebook.google.com.
Windsurf: windsurf.com 308-redirects to devin.ai/desktop. It's a Cognition product now.
Grammarly: its own site says Grammarly is now a part of Superhuman, and the footer reads 2026 © Superhuman Platform.

Vendor documentation moves too, which breaks the citation trail in older articles: platform.moonshot.ai → platform.kimi.ai, llama.com → developer.meta.com/ai, docs.claude.com → platform.claude.com, and OpenAI's model index from platform.openai.com/docs/models to developers.openai.com/api/docs/models.

That gives you a cheap diagnostic. A 301 or 308 on a vendor's own domain is the single highest-signal tell that something changed. Companies redirect when they rebrand, get acquired, or consolidate — and they do it long before the listicles notice, because the redirect is the thing they have to ship and the announcement is optional.

curl -sI https://windsurf.com | head -2

Thirty seconds per tool, and it catches the failures no amount of careful writing will.

How to check anything in about a minute

Go to the vendor's docs, not their marketing site. The models page and the pricing page are maintained by people who get paged when they're wrong. The homepage is maintained by marketing.
Follow the redirects first. If the docs URL you have moves, treat everything downstream of it as suspect.
Read the pricing page even if you don't care about price. It's usually the most current list of what actually exists, because billing has to be right.
Write down the date you looked. This is the part almost nobody does, and it's what separates a claim a reader can evaluate from one they have to trust. 1M context window (vendor docs, 15 August 2026) ages honestly. 1M context window does not.

That last habit is the whole discipline. A dated claim that's six months old is still useful — the reader knows exactly how much to discount it. An undated claim that's six months old is indistinguishable from a current one, which is worse than saying nothing.

The current lineup

Read from each vendor's own documentation on 15 August 2026. Everything here will decay; the date is so you know how far.

Vendor	Current	Notes from the vendor's docs
OpenAI	GPT-5.6 Sol	Frontier model for complex professional work, 1.05M context. Terra balances cost; Luna targets high volume
Anthropic	Claude Opus 5	1M context, 128k output, $5/$25 per MTok, May 2026 training cutoff. Sonnet 5 at $2/$10; Fable 5 above both
Google	Gemini 3.7 Flash	Leads the stable line. 3.1 Pro is preview
xAI	Grok 4.6	500k context, knowledge cutoff 1 Feb 2026. grok-4.3 has 1M
DeepSeek	DeepSeek V4 Pro	1M context, 384K max output, $0.435/$0.87 per MTok. V4 Flash at $0.14/$0.28
Meta	Llama 4	Still current
Mistral	Mistral Large 3	State-of-the-art, open-weight, general-purpose multimodal
Moonshot	Kimi K3	1M context, built for long-horizon coding and end-to-end knowledge work
Alibaba	Qwen3.7-Max	Model Studio exposes OpenAI-, Anthropic- and DashScope-compatible endpoints
Cohere	Command A Plus	128k context, 64k output, vision input

One pricing note worth having: from 16 August 2026 DeepSeek moves to peak and off-peak billing, off-peak at half the peak rate, with peak defined as 01:00–04:00 and 06:00–10:00 UTC. If you run batch workloads, scheduling them outside those windows halves the bill. That is the kind of detail that exists only on a pricing page and appears in no comparison article anywhere.

What to actually do with this

If you're choosing a model, don't start from a comparison article, including this one. Start from the vendor's docs for the two or three candidates you already suspect, and use articles to work out which candidates to consider. Our AI tool finder is built for the shortlisting half of that problem and makes no attempt to replace the docs.

If you're maintaining a system, the useful move is to write down which model version you're pinned to and when you last checked it. Most teams can tell you which provider they use and not which model — and the gap between those two facts is where the surprise bills and the silent quality regressions live.

If you're writing about this, date your claims and re-read your sources before you refresh a post. The reason so much AI content is wrong isn't that writers are careless. It's that the format encourages you to update the publish date without updating the page.

And if you're running a directory: ours drifted this badly in months, with a recommender attached that was quietly ranking superseded models above their own replacements. Nothing about the software was broken. The data just aged, silently, because nothing was designed to notice. That's the failure mode to build against — not incorrect entries, but correct entries that stop being correct while everything continues to look fine.

Sources

All read on 15 August 2026:

Anthropic model overview — Opus 5, Sonnet 5, Fable 5 current; Opus 4.8/4.7/4.6 and Sonnet 4.6/4.5 under Legacy models; context windows, max output, pricing and training cutoffs
OpenAI models — GPT-5.6 Sol as Frontier model for complex professional work, 1.05M context, plus Terra and Luna
xAI models — grok-4.6 at 500k, grok-4.3 at 1M, knowledge cutoff February 1, 2026
DeepSeek pricing — deepseek-v4-pro and v4-flash, 1M context, 384K max output, and the peak/off-peak change effective 16 August 2026
Gemini API models — the stable/preview split across the Gemini 3 line, and the Nano Banana image models
Mistral models overview — Large 3, Medium 3.5, Small 4
Cohere models — command-a-plus-05-2026 at 128k/64k against command-a-03-2025 at 256k/8k
Kimi platform docs — kimi-k3 at up to 1M context
Alibaba Model Studio — qwen3.7-max and the current Qwen line