Seven questions every AI tool vendor should answer before you renew
A practical checklist for your next vendor review — the questions that separate "the team loves it" from "the team is paying for it and quietly ignoring it."
By Colin Cardwell
Your AI tool vendor has a smoother renewal conversation than you do. That's because they've had the conversation more times.
Every AI tool vendor's renewal playbook looks similar: show up a month out, walk through a pre-baked "success story" deck, ask if there's anything to iron out, and put the contract in front of you before the quarter closes. The deck is glossy. The numbers are flattering. The default action is to renew.
It's your job to make that default harder. Below are seven questions we've found separate the tools your team genuinely uses from the ones you're paying for out of inertia. Ask them at renewal — in writing if you can — and see which vendors answer crisply and which ones pivot to a demo.
1. What's our real adoption rate?
Not "licensed seats". Not "signups". The percentage of our licensed seats that have an active, sustained usage pattern — ideally measured weekly over the last quarter.
Any vendor's admin portal can show you this, but the numbers they'll put on their deck will be the softest definition they can get away with (logged in once, or activated account). Pin them down on "at least one meaningful session per week for the last 8 weeks" and watch what happens to the number.
2. What's our hours-saved per active user per week?
Normalised. Total hours saved is impressive but meaningless — 500 hours saved in a 10-person team is stellar, 500 hours in a 500- person team is a rounding error. The comparable number is hours saved per active user per week.
As a rough rule of thumb from everything we've seen at AiGILE: under 1 hour/user/week is disappointing, 2-3 is healthy, 5+ is genuinely transformative and you should be buying more licences.
3. Which features are being used, which are ignored?
Good vendors can tell you. Great vendors can show you a breakdown by seat. This matters for two reasons:
- If 80% of your usage is one feature that other tools also do, you're paying for a premium suite to get commodity value. There may be a cheaper tool.
- If a feature the vendor charges extra for is sitting unused, you have negotiating leverage at renewal. "You're charging us for feature X and the data shows nobody's touching it — either prove its value or drop the price."
4. How does our usage compare to similar organisations on your platform?
Benchmarks are the single piece of data the vendor has that you don't. They see across hundreds of customers. Make them tell you where you sit. Are your power-users above or below average? Is your adoption curve trending like healthy customers' or like churn-risk customers'?
Good vendors have this data ready. If they push back with "we don't share aggregate data", it's either genuinely unavailable (unlikely) or they don't want you to know you're underperforming (more likely).
5. What onboarding and enablement are you committing to this renewal?
Renewals aren't passive. The tools that thrive in your org thrive because someone — often the vendor's CS team, ideally — keeps reintroducing them to new hires, new teams, new use cases. Most vendors give you white-glove onboarding in year one and leave you to fend for yourself afterwards.
Ask what they're committing to in year two onwards. Quarterly training? Monthly lunch-and-learns? A dedicated CS contact? Feature-release walkthroughs? If they can't name a specific deliverable, you'll be paying the same price for significantly less support.
6. What's your roadmap for the issues our team has flagged?
This assumes you know what your team has flagged — which is where a measurement tool like The GAiGE or a well-run quarterly retro earns its keep. If you can walk into the renewal with a list of specific, dated complaints from your actual users, the vendor has to engage with them.
Watch carefully how they respond. "That's on the roadmap for Q3" is fine. "That's interesting feedback, we'll look into it" is a brush-off. "No one else has raised that" is a lie — or it's a real differentiator in how your team uses the tool, in which case it's absolutely on the vendor to solve.
7. If we cancelled today, what would we lose?
The single sharpest question. Makes the vendor articulate the specific value that would disappear from your team's week if the tool vanished overnight.
A good vendor will have an answer involving specific workflows and specific outcomes: "Your engineering team is saving ~6 hours/week on code review. Your marketing team generated 40 campaign variants last month they wouldn't have otherwise."
A struggling vendor will give you a feature list. Features are inputs. Outcomes are what you're paying for.
A note on attitude
None of this is adversarial. Good vendors want these questions — they're selling into senior buyers who already ask them, and if you don't, they have to waste time guessing what actually matters to you. Asking them clearly and early improves the renewal conversation for everyone.
What it does do is pull the conversation out of the vendor's pre-baked deck and into your reality. That's the whole game.
If you're evaluating how your team's answers compare to vendor claims, The GAiGE is built for exactly this job. Happy to show you what it looks like against your current AI stack.
Want more like this?
Getting the Measure of AI. AI Impact Measurement news when it's fresh. One click to unsubscribe.