Not your vendor. Not the benchmark you were shown. Not, if you ask them plainly, your own team. oneground measures it — on your documents, against the right answers — and gives you the evidence rather than an opinion.
One setting decides whether you store your data once, or four times over.
Every dot is a real document. Move the control and watch the copying happen. Positions in the picture are an illustration; the counting is done in full. These 150,000 documents are public ones, not yours — measuring your own is what the tool does, from the command line.
You know how the system performed in the vendor's benchmark, on the vendor's data. Your documents are not that data. They are longer, or shorter, or more repetitive, or all about the same eleven topics. Whether the setup you are paying for suits them has never been measured — because measuring it meant knowing the right answer for thousands of real questions, and nobody had that.
Trying a different setup means standing it up, loading your documents, writing the tests, running them: a week of engineering to answer a question that might come back "no change." So it isn't done. The first choice becomes the only choice, and stays load-bearing for years.
Ask your team why this system, this configuration. You will get a blog post, a benchmark on someone else's data, and a name. The reasoning lives in people's heads and leaves when they do. When quality drops, or the bill needs defending, there is nothing to show.
None of this is carelessness. The measurement was impractical. That is what changed.
Give it a sample — ten to twenty thousand documents, not the whole archive. It works out the right answers by exhaustive search, then measures how well each realistic setup actually finds them. Not a proxy and not a leaderboard: your data, scored against truth.
What you hand it, concretely: either the vectors you already have, or your text plus the name of the model you embed with — pinned, so the report records which model produced the numbers. Plus a set of queries: real ones from your logs if you have them, which is better than invented ones and is recorded either way. If you have neither a sample nor vectors, it will still do capacity arithmetic from what you declare — and it will refuse to issue verdicts from declarations, because a declaration is not a measurement.
Three architectures and many configurations, all scored the same way on the same sample, in minutes on a laptop. Speed is the one thing that needs a real system running, and for that it uses one, under real load, on your own hardware or on a machine you rent for about the price of a coffee.
Not a summary — the evidence. What was measured, with which settings, on which sample, producing which number. What couldn't be measured, named rather than quietly filled in. And the tool's own accuracy, dated: every report says when oneground last checked itself against published results, and by how much it was off.
That last part is the difference between a benchmark and an audit.
What it turns up is rarely what you would guess. On one public collection, the clever architecture everyone reaches for scored worse than the simplest one — and cost nearly four times the storage. On a second collection the ranking held, but the answers went stale in the opposite direction. One set of documents is never the whole story, and now you can see which story yours tells.
Most tools in this space are built to produce a recommendation. This one is built to produce a defensible one, which means it will sometimes decline:
No option meets every constraint, so nothing is recommended.
the most common first result, on a real corpus · the report it came from is publishedIt won't call a coin flip a pass. The same setup measured twice came back at 38 milliseconds and 43, against a 40-millisecond target: one run says pass, the other says fail. It reports the spread and withholds the verdict.
It won't compare things that weren't measured the same way — two systems on two machines are not comparable, so it refuses, and tells you exactly what to run to make them so.
It won't measure your network and call it your database. When the round trip is most of the number, it says so rather than reporting a figure that is mostly your own connection.
A tool that always answers is a tool you cannot cite. This one can be cited, because the places it declines are marked.
| the question | what oneground gives you |
|---|---|
| How does our AI search perform on our own documents? | A measurement of your sample against the right answers, not a vendor benchmark. |
| What would a different setup have done? | The realistic alternatives scored on that same sample, in minutes. |
| Why did we choose this, and can we defend it? | A record of what was measured, what wasn't, and how accurate the tool itself is. |
Free and open source, permanently. The analysis runs on a laptop. A speed test under real load runs on a machine you rent, under your own account, for about the price of a coffee — and the tool shows the price, asks before spending, and shuts the machine down itself.
Nothing. No upload, no account, no telemetry, no oneground server. For the rented-machine speed test, your sample goes to a machine you rent and comes back as a file.
Before committing to a vector database. After a migration, to show the new setup beats the old one. When search quality drops and nobody can say why. When the infrastructure bill needs defending.
It measures retrieval — which documents come back — not the answers your AI writes on top of them. It works from a sample, not your whole archive, and says so. It will not rank systems it has not actually run. And it will not produce a number it cannot stand behind.
It does not measure chunking — how your documents are cut up before they are indexed. That is named here rather than left for you to find out, because chunking is arguably the decision that changes retrieval most, and a tool that measured everything except the biggest lever would be worth knowing about in advance. It is on the roadmap and no code implements it today, so nothing on this page depends on it.
Send your engineering lead this page. To use it, one command:
pip install oneground # Python 3.12 or newer
To check us before you use it — which is the point of the project — clone the repository and run this. It is standard library Python: no install, no account, no downloads beyond the clone itself.
# recompute the figures on the lab page from the data that page ships
python site/teaser/verify_teaser_data.py
It re-derives the numbers independently of the page and exits non-zero if any of them
disagree. There is a second script beside it, check_hosted.py, which asks whether
the copy of the lab served from this site is byte-for-byte the copy in the repository.
Re-verifying the whole 150,000-vector fixture is a heavier job, and we would rather say so than let you discover it. It takes a checkout of the repository and a 461 MB release asset, and about ten minutes; the one-line command you may have seen elsewhere fails from a fresh install today, with an error that does not say why. The sequence that works is set out step by step, as oneground documents it.
For the engineer who will run it How it works → — the four commands, the measurement method, the full refusal output, and how to check the figures on this page against the data they came from. For everyone The lab, live → — 150,000 real documents, one query followed all the way through, and the verdict at the end of it.Today: measure your documents, compare the setups, test against a real system, and get the decision with the evidence behind it.
Next — already built and already in the public repository today: a second database with a head-to-head on one machine, speed tests run three times with the spread reported, a measurement of how much traffic your setup takes before it breaks, a second reference collection, and a warning if your own documents are being silently cut off before they are ever searched — a quiet and common cause of bad results. Shipped when it is tested; dated when a witnessed run makes it true.
We are naming it now because it is in the repository for you to look at today. We would rather be checkable than impressive.
On 15 September, on a machine that is not the developer's, a clean clone of the repository was
checked against its own published values. Every digest reproduced. The figures on this page were
recomputed from base.bin and agreed: storage amplification 3.7152 against a
published 3.715, boundary crispness 0.0362 against 0.036, p99 copies 4, the copies
histogram 3.6 / 5.6 / 6.5 / 84.3%, and 144,563 vectors copied. All 2,000 held-out
queries agreed with the routed-region recall, with zero disagreements. The run reported
ALL CHECKS PASSED.
oneground is the Choose door of the oneproof suite. Measured instead of asserted — from the architecture you choose before you build, to the evidence an answer rests on, to the authorization an action required.