How We Verify a Model Is Real: Full Methodology
Here's every step the verification tool actually runs, laid out in full: what each of the 11 checks looks for, how a verdict gets scored, and an honest account of where we know this tool's limits are.
A. All 11 checks, listed in full
This is the complete set of checks the backend actually runs — not one more, not one less — and it matches the code exactly.
Connectivity
Whether the endpoint responds at all, and whether the key has access to the requested model. This is the baseline — nothing else can run if this fails.
Model name echo
Whether the model field in the response matches what you requested. If it's echoed but doesn't match, that's suspicious; not echoing it at all isn't a problem on its own — some legitimate providers simply don't return this field.
Billing metadata
Whether prompt_tokens / completion_tokens under usage follow the standard spec. Missing fields aren't flagged as a fault — marked "not tested" instead, since usage is an optional field plenty of legitimate providers skip.
Response structure
Whether required protocol fields like choices / message are all present, with support for streamed (SSE) responses — some providers default to or force streaming, so we reassemble the equivalent structure first before judging, to avoid false positives.
Model self-identification
Directly asking the model what it is and who built it. Requesting Claude and getting a response that claims to be GPT, or another vendor entirely, is the most direct evidence of substitution.
Precise computation (dynamic question)
A randomly generated arithmetic problem each time (a 4-digit number times a 3-digit number, plus a 4-digit number) with one verifiable correct answer — impossible to game with a cached or preset response. Weaker substitute models tend to get these wrong.
Context integrity
Plants a random verification code near the start of a roughly 2,000-word passage, then asks for it back at the end. Failing to recall it can indicate the context got silently truncated.
Token-count sanity check
Compares the actual character count of the input against the prompt_tokens the endpoint reports. A significant mismatch in scale can point to billing irregularities.
Self-reported knowledge cutoff
Asks the model to state its own training data cutoff. Requesting the latest version and getting a self-reported cutoff from 2023 or earlier can mean it's been swapped for an older model.
Instruction-following precision
A tightly formatted instruction (output exactly three lowercase, comma-separated words, no spaces or extra punctuation) — whether the model follows it exactly is an objective read on capability.
Cache-hit behavior
Sends the same prompt twice in a row and watches the cache-related usage fields and the change in response latency. A cache hit is a positive signal; no hit doesn't mean anything's wrong — plenty of relay channels simply don't support caching.
B. Scoring rules
Pass
This check looks normal
Inconclusive
This check didn't line up
Failed
Clearly abnormal
Not tested
Field missing / can't determine
- "Not tested" never counts against the verdict. Plenty of legitimate providers simply don't return certain optional fields — that's not evidence of fraud, so this category is excluded from the final scoring.
- The overall verdict is scored by how many checks come back inconclusive: 0 = no issues found; 1-2 = inconclusive; 3 or more is required before we flag it as failed.
Design principle: we'd rather miss a real problem than falsely flag a legitimate provider. A single test run always has some noise — network hiccups or a model occasionally misreading an instruction can make one check fail on its own — so we deliberately set a higher bar before returning a "failed" verdict.
C. Our position (an honest statement)
The check process never stores your API key — not logged, not written to any database.
We're also a relay directory with paying advertisers listed on the site — but the verification logic is applied identically to every provider, and never adjusted for a commercial relationship.
A result only reflects what we observed in that one call — it's not a legal determination about how that provider operates its business.
A single result carries real noise — we recommend retesting at different times and judging the trend, not any one run.
D. Academic backing
The problem of relay APIs quietly substituting models isn't something we made up — it's been documented in public academic research:
"Real Money, Fake Models: Deceptive Model Claims in Shadow APIs"
Authors: Yage Zhang, Yukun Jiang, Zeyuan Chen, Michael Backes, Xinyue Shen, Yang Zhang (CISPA Helmholtz Center for Information Security) · March 2026
- · 45.83% of fingerprint tests failed model identity verification
- · Performance deviation reached as high as 47.21%
- · Covered 17 relay API services that had collectively been cited by 187 academic papers
- · The paper also found significant unpredictability in these services' security behavior
Our verification tool is an independent implementation and isn't affiliated with this research — we cite it only to show the problem is real and has a public academic basis.
E. What we can't do
Intermittent substitution
A provider that only swaps to a cheaper backend during peak hours and behaves normally the rest of the time won't be caught by a single test run.
Subtle capability degradation
Minor quality drift within the same nominal model (say, quietly swapped for a distilled or quantized variant) is beyond what our current checks can distinguish.
Output style comparison against an official baseline
This kind of comparison needs a genuine official-API baseline sample to compare against, and we don't maintain one, so we don't attempt it.
Our automated checks don't cover these cases — pair the tool with the manual self-check checklistfor a fuller picture.
FAQ
How do you actually determine whether a model is real?
Using the endpoint and throwaway key you provide, we send several real requests on the spot and check connectivity, model self-identification, precise computation, context integrity, and other hard indicators across 11 checks — each one judged independently, with no subjective scoring involved.
What exactly are these 11 checks?
Connectivity, model name echo, billing metadata, response structure, model self-identification, precise computation (dynamic question), context integrity, token-count sanity check, self-reported knowledge cutoff, instruction-following precision, and cache-hit behavior. Section A on this page lists exactly what each one does.
If the result comes back "inconclusive," does that mean the provider is faking it?
No. A single test run carries noise — network hiccups or a model simply misreading an instruction can make one check fail on its own. Our scoring rule flags 1-2 failed checks as "inconclusive" and only escalates to "failed" at 3 or more, by design erring toward missing a real problem rather than falsely flagging a legitimate provider.
Do you store my API key during the check?
No. The key is used only for that single verification request — it's never logged or stored anywhere, and it's gone as soon as the request completes. Even so, we recommend using a throwaway key and deleting it from the provider's dashboard once you're done testing.