How to check if an AI launch claim holds up

An AI launch post is written before almost anyone outside the company has used the product. That doesn’t make it wrong. It means the post is a list of promises, and the only way to know which promises hold is to wait for people to use it and then read what they say. This checklist is the method we use for every Day 7 receipt, written so you can run it yourself on the next launch you care about.

Each step comes with an example from a receipt we have already stamped, so you can see what it looks like in practice. The full rules are on our method page.

1. Go to the official launch page and quote it exactly

Start with the vendor’s own announcement, changelog or release notes, not press coverage and not a summary on social media. Copy the claims word for word. Paraphrasing a claim is where most arguments about launches go wrong: “up to 8x faster” and “8x faster” are different promises.

Read every official page, not just the main one, because they don’t always agree. For Gemini 4 Argon, Google’s main launch post said the model was coming to Google Antigravity “for Google AI Ultra subscribers”, while the Antigravity post said: “Today, we’re rolling out Gemini 4 Argon directly inside Google Antigravity for Google AI Pro and Ultra subscribers”. Pro subscribers then filed busy forum threads saying it was missing. When two official pages disagree, quote both and test each.

2. Separate claims you can check from claims you can’t

Most launch posts mix five kinds of claim: availability (“available today to…”), price, speed or cost improvements, behaviour promises, and headline features. Some of these can be checked by users within days. Others, such as benchmark rankings and list prices, usually can’t be confirmed or disproved by ordinary use.

Take GPT-6.1 Sol. OpenAI wrote that it “nearly matches GPT-6 Astra’s intelligence on agentic coding, computer use, and professional work at one-fifth of Astra’s standard input and output token prices.” We found no solid user comparison either way, so that claim is marked unclear. The claim that it was “available starting today to all Plus, Pro, Business, Enterprise, and Edu users in ChatGPT Work and Codex” was checkable, and it came out mixed: paying users could not see the model until they updated their apps.

Marking a claim unclear is not a soft verdict. It is an honest statement that the evidence doesn’t exist yet.

3. Test “available today” against the people who should have it

Availability is the easiest claim to check and the one that fails most quietly. Look for forum threads and issues from people on the named plans who can’t find the feature. Then look for the opposite: people on those plans who are using it.

Both happen. Cursor Projects said “Projects are available in beta and rolling out to all users starting today”, and that claim holds: people on individual plans are using it, and the one access problem we found, on iOS, was fixed within a day. Gemini 4 Argon’s Antigravity rollout is the counter-example: Pro users reported it missing from 13 to 15 October, and some paying Pro and family-plan members still reported it unavailable days later.

4. Read the price claim the way you will be billed

A price per million tokens is only part of what a model costs you. What matters is the cost of finishing your task, and that depends on how many tokens the model uses and whether discounts such as caching actually apply.

Gemini 4 Argon launched “with cached input tokens at 95% off”. A developer sent the same long prompt prefix repeatedly and got zero cached tokens, while an older Gemini model cached the same prefix. Google staff replied that explicit caching is not supported for this model and said they were looking into it. On paper the discount exists; in that developer’s bill it didn’t.

Claude Opus 5.5 shows the other side of the same problem. Anthropic said it “costs 40% less to run than Opus 5”. The list price cut is real, but one detailed Claude Code report measured about twice the thinking tokens and 1.6 times the output per request from 1 October. If that holds more widely, it would eat into the saving. Nobody has published an independent cost study, so we left the claim mixed rather than failed.

5. Only count reports you opened, and weigh where they came from

A solid report is one you opened yourself, written by someone who used the product, specific enough to check, and about this launch. A search-result snippet is not a report. A tweet saying “this is amazing” is an opinion, not evidence. A GitHub issue with steps to reproduce is evidence.

Be honest about bias in your sources. GitHub issues and support forums lean negative, because people post when something breaks. Hacker News threads are mostly opinion, with the occasional detailed first-hand comment. Hugging Face discussions are useful for open-weight models because people post measurements there. We note the source mix on every receipt and look for praise threads as well as bug reports.

6. Count before you judge, and say “too early” when you must

The loudest launches are often the hardest to judge, because noise arrives before experience. Our rule is that fewer than about five solid reports means Too early, however big the launch was.

Muse Spark Open drew a large Hacker News thread, but its headline claim was that it is “the first model in its class to run on your desktop using two Apple M5 Ultra Mac Studios with the latest MLX framework.” When we last checked, Apple’s own page still listed the 512GB M5 Ultra option as coming in late October, and no independent two-machine run had been reported. So the receipt stays Too early. Cursor Projects is the same: its headline promise is that it “maintains context over months of work, delegates tasks to thousands of subagents, and performs recurring work without being prompted”, and no report yet tests any of that at scale.

7. Separate the product from the tools around it

Some failures belong to the launch, and some belong to the ecosystem. If a model works well but a third-party runtime can’t load it yet, that is worth reporting, but it says something different from the model failing.

EmbeddingGemma 2 is the clearest example we have. Google said: “We worked closely with the following partners to ensure EmbeddingGemma 2 works immediately where you build”. That claim came out mixed: Ollama’s build would not install on Linux or Windows, vLLM failed to start it on some GPUs, and the Quickstart missed a dependency. But the model’s own claims, open weights, a shared embedding space across text, images and audio, and better code retrieval, held up in independent tests. We counted the tooling gaps against that one claim and stamped the model Delivered.

The checklist in short

1. Quote the official launch page word for word, and check every official page that makes the same claim.
2. Sort claims into checkable (availability, behaviour, features) and hard to check (benchmarks, list prices).
3. Test availability against people on the named plans, both missing and working.
4. Judge price by the cost of finishing a task, including token use and whether discounts apply.
5. Count only reports you opened that are first-hand and specific, and note which way each source leans.
6. With fewer than about five solid reports, say it’s too early.
7. Keep tooling and partner problems separate from the product’s own claims.

Every receipt in our archive follows these steps, with each claim and each report linked so you can check our work. If you think we got one wrong, the corrections page explains how to tell us.