We test AI tools the way a business would use them. Each test starts with a real task, such as answering support emails, cutting a long video into clips or processing invoices, and ends with the numbers a buyer needs: how long it took, what it cost and what went wrong.
We set the rules before we start
Before a test begins, we write down what we are comparing and how. Every tool gets the same input. We use the plan and price shown on the tool’s own pricing page on the day of the test, and we give that date. We decide in advance what counts as a usable result, so a tool cannot be judged on a softer standard after the fact.
Two kinds of test
AI tools and services. Commercial tools are tested in the tool’s own service, on the paid plan a business would buy. We run each one through the task ourselves and record the session. We time each step from start to finish, note what it cost on that plan, and keep every output, including the ones we would not use.
Open-source and open-weight projects. Models and tools that anyone can download are tested on our test machine, so the numbers describe what you would get on your own hardware. We install them ourselves, run the same task, and measure speed, memory and the quality of the output. Where a project sends data anywhere, we check what it sends.
Either way, the figures in a piece come from our test record. If a number is not in our test log, it is not in the piece.
A person reviews every result
AI tools do much of the work in a test, because that is what we are testing. The judgement stays with us. An editor reviews every output by hand, checks every figure against the test log and decides the verdict. Where AI helps draft a write-up, it works from our test notes, and the editor edits it and checks every claim before it is published. The line at the end of each piece, “How this was made”, says who directed the test and who checked it.
What we will not do
- We give no verdict on a tool we have not run. If we have not tested something, we say so.
- We do not use another site’s numbers as our results. A maker’s own figures appear only as the maker’s claim, named as such.
- A company cannot influence a result.
When things change
AI tools change their prices, plans and features often. When a retest changes a finding, we update the piece and note what changed and when.
Spotted an error?
Tell us at our contact address. If we got something wrong, we correct it and say so on the page.