The Dealer's AI Fast-Track › Part 5 of 11
Part 5
Every vendor deck you see this year will say AI on it. These eight questions separate the tools from the stickers. A good vendor answers all eight crisply; a repackaged vendor gets vague by question three.
If the answer is a feature list you had in 2022 with "AI-powered" in front of it, it's a sticker.
Canned demos hide everything. Any serious vendor can run a pilot on your real leads or ROs.
Every AI system makes mistakes. A trustworthy vendor has an error story: escalation to humans, review queues, correction logs. "It doesn't make mistakes" ends the meeting.
Get it in writing. Your customer and pricing data should not be improving a product your competitor buys.
Undisclosed AI conversations are a trust risk and, increasingly, a legal one. You want disclosure built in.
"Engagement" is not a result. Appointments set, calls answered, hours saved — numbers you can audit against your own systems.
Spoiler: you are — regulators treat claims made by your tools as your claims. So the real question is what the vendor does to prevent it and how they indemnify you.
Month-to-month after a pilot, your data exportable, no hostage-taking. AI tools are evolving too fast for three-year lock-ins.
Print it: the eight questions and five contract terms on one page (PDF). Bring it to the next vendor meeting.
The eight questions interrogate the vendor. Once a tool survives them, switch lenses entirely: stop evaluating it like software and start evaluating it like labor. This isn't a slogan — it's how we run our own companies. Before DealerSpark sold anything to a dealer, our founder spent three years building an AI workforce inside a nationwide collateral-recovery firm — recorded calls, sensitive financial data, lender compliance, real consequences for mistakes. That company runs about 14 human employees today, with roughly two dozen AI employees working alongside them. Some have big jobs, some have tiny part-time jobs — and every one of them is managed with the same six questions you'd ask about a human hire:
| Question | What you're asking |
|---|---|
| Job | What, exactly, is it responsible for? (The narrower the job, the better the work — no human runs the whole store either.) |
| Quality | Is the work good enough for this job? Perfect is the wrong standard — you've never managed a perfect human. Reliable at the level the job requires is the standard. |
| Output | How much work actually gets done? Count it the way you'd count a person's. |
| Supervision | How much oversight does it need? Higher-risk job = tighter controls and more human review (Parts 6 and 9). |
| Wages | What are you paying it, all-in, per month? |
| ROI | What is the work worth against those wages? |
Notice what's not on the list: parameters, benchmarks, model names — the things AI vendors love to talk about. You're not hiring artificial intelligence. You're hiring somebody to do a job, and the whole evaluation comes down to one question: can it do the job? This lens works whether you're buying a vendor's agent, building your own (Part 9), or deciding whether a $20 assistant seat is earning its keep.
Here's the operator's test as a worksheet. Pick one job you're thinking of handing to an AI tool, fill in your own numbers, and it works out what you're paying per unit of usable work and whether the hire earns its wages. The starting numbers are an illustration (an after-hours service-call agent), not a benchmark. Replace every one of them.
Everything is per month unless it says otherwise. Nothing you type here is sent anywhere; the math runs in your browser.
6 · ROI
How it figures: AI total cost is its wages plus the supervision time (hours per week × 4.33 weeks × the reviewer's hourly cost). Good units are output × the quality share. Net is the value of the good units minus the AI's total cost. The human figure is there for comparison only; for a lot of AI jobs the honest baseline isn't a person, it's the work not getting done at all, which is why the value-per-unit line matters more than the wage comparison.
The Dealer's AI Fast-Track: all parts