AI vendor scorecard: compare vendors on the same ten criteria.
Name up to three AI vendors, weight ten fixed criteria for your job, and score what each one showed you, from no answer to checked by you. You get a total out of 100 and the must-have gaps for each. Nothing is pre-scored, stored or sent.
What does the AI vendor scorecard do?
The AI vendor scorecard is a free browser tool for comparing up to three AI vendors on ten fixed criteria, with weights you choose. Score each vendor from 0 (no answer) to 4 (in writing and checked by you). It totals each vendor out of 100 and lists any must-have gaps. Print it for your buying team.
How do your vendors compare on the same criteria?
Name up to three vendors, set a weight for each criterion, then score what each vendor showed you.
- Scale
- 0No answer, or not offered
- 1Answer, but vague
- 2Clear answer, not in writing
- 3In writing
- 4In writing and checked by you
| Criterion | Weight |
|---|---|
| 1. Fit to your jobDoes the product finish your one job end to end, to a done state you could check?Ask: Can you walk us through our job, start to finish, and show where it stops? | |
| 2. Product stageIs it available today, in a pilot with customers, or still being designed? Score what they say in writing.Ask: What stage is this product at, and how many customers use it in production today? | |
| 3. Evidence it worksDated, named evidence on work like yours: references you can call, a test record, a case with permission.Ask: Who can we speak to who runs this on work like ours, and since when? | |
| 4. Measured on your dataWill they measure accuracy on your own data with a stated method, before you rely on it?Ask: How will you measure results on our data, and what happens to the misses? | |
| 5. Tested connectionsNamed connections to your systems, each with the actions tested (read, write) and the date.Ask: Which of our systems have you connected to, with which actions, and when was it tested? | |
| 6. Human controlYou choose what needs a person’s approval, and you can stop it yourself at once.Ask: Which actions need our approval, and how do we switch it off? | |
| 7. Records you can reviewEvery action logged, reviewable and exportable by your team.Ask: Can we see and export a log of every action it took, and who approved it? | |
| 8. Data handling and securityWhere data is stored, for how long, who processes it, whether it trains models, and certificates with their scope.Ask: Where is our data kept, who are your subprocessors, and is our data used to train models? | |
| 9. Pilot and contract termsScope, measures, price, end date, support contact and exit terms, agreed in writing before work starts.Ask: What is in writing before we start, and how do we stop if it does not work? | |
| 10. Total cost at your volumeEvery cost at your volume: subscription, usage, onboarding, third-party fees and people time.Ask: What would we pay in year one at our volume, including fees from other providers? |
“Checked by you” means you saw it on your own data, read the document or spoke to a reference.
How should a buying team use the scorecard?
- 1Agree the weights first
- 2Ask every vendor the same questions
- 3Score only what you saw
- 4Read the gaps before the totals
What does the scorecard assume?
- One job at a time
- Same questions for everyone
- Your weights are yours
What can’t the scorecard tell you?
- Whether a vendor is honest
- How the product performs
- Legal or security sign-off
- Which vendor to choose
Last updated
Bring us one process. We’ll show you how it would run.
A 30-minute call with a specialist: what it connects to, what a pilot involves, and an honest view on fit, including when it isn’t one.
- 30 minutesOne call, one process
- No commitmentNo sales script
- 4 business hoursTypical reply time · Mon to Fri, 7:00 to 22:00 IST
- 1You get a short agenda by email.
- 2We look at one process with you and how it would run.
- 3You get an honest view on fit, and what a pilot would involve.