The 2.5× rule: how we keep ROI estimates in line with the evidence
How The GAiGE calculates ROI defensibly: the 2.5× extrapolation cap, the use-your-own hourly rate, aggregation thresholds, and where you should push back on our methodology.
Colin Cardwell
Every AI vendor has an ROI calculator. Most of them produce results that would embarrass a mid-career finance person. "Save 14 hours per user per week!" Not if the user only works 37 hours a week and spends a third of them in meetings, they won't. The gap between those numbers and reality is where trust in AI measurement is lost.
The GAiGE exists because we'd rather be quietly useful than impressively wrong. This post is a walk-through of how we calculate the numbers you'll see in the product, and the decisions we've made to keep them defensible. If you're evaluating whether to trust our reports, this is the page to start with. If you're a sceptic, we're hoping to earn about eighty percent of your trust by the end of it, and leave you with useful questions for the last twenty.
Where most AI ROI numbers go wrongLink to this section
Four common failure modes, in roughly decreasing order of frequency:
- Unbounded extrapolation. A small number of users report saving an outlier amount of time. The calculator multiplies that across the whole organisation. Twelve months of "savings" are invented in a spreadsheet.
- Survivorship bias in the respondents. Only enthusiasts reply. Their answers get treated as representative. Everyone who quietly ignores the tool is invisible to the number.
- Conflict of interest. The party calculating the ROI also wants the renewal to go through. Guess which way the ambiguous decisions break.
- Wrong unit of measurement. "Hours saved" with no hourly rate, or with an hourly rate pulled from thin air. Impressive-looking numbers that nobody can turn back into dollars.
These aren't strawmen. We've seen all four in vendor decks, and we've caught ourselves drifting towards a couple of them during product design. Naming them helps.
The 2.5× ruleLink to this section
Here's the cap that gives this post its name.
When someone answers a pulse and says a tool saved them, say, "1-2 hours today", we have a number to work from. If they answer a few pulses a week, we have a few numbers. The temptation is to take the average, multiply it by every working day of the year, and call it an annual saving. Nobody working with real data thinks this is a good idea.
What we actually do:
- We never scale anyone's reported time up by more than 2.5×. If someone answers once a week, we scale what they told us to cover the rest of the week by no more than 2.5 times, not five to seven. Nobody is credited with saving more in a day than their working day holds, and reported hours always sit beside the estimate.
- We never report extrapolated numbers without a clearly labelled reported figure next to them. The reader always sees both. If the two diverge a lot, the reader knows to be careful.
- We show the organisation's response rate beside every figure. A 4.5/5 average from 80% of your team is a different fact from a 4.5/5 from 15%.
- On an anonymous pulse, every figure needs answers from at least five people before we show it. With fewer, a figure could point to the people who gave it.
Is 2.5× the right cap? It's a deliberately conservative choice rather than a figure from research. If you want to argue for 2× or 3×, we'll happily have that conversation. What we won't do is claim a uniformly applied 10× because it makes the deck look better.
The blended hourly rate: yours, not oursLink to this section
The other number that turns hours into dollars is the hourly rate. We could make one up. Most ROI calculators effectively do: they multiply by "the average knowledge worker salary" and hope you don't notice.
Our approach is boring: you set it. On Pro, you set it in Settings, as a fully loaded figure (salary plus on-costs, divided by productive hours a year). Until it's set, we show hours saved and hide the dollar figures. It's your number. If your auditor has a different view, you can argue that with them. We just do the multiplication.
The same principle applies to the tool cost side of the equation. You enter what you actually pay per seat per month, including any negotiated discounts and committed minimums. We don't pull list prices from the vendor's website.
Anonymous pulses, because candid answers depend on themLink to this section
One more principle, and it's a product decision as much as a methodology one. Pulses are anonymous by default, and every pulse says before its first question whether it is anonymous or identifiable. When a pulse is anonymous, your name isn't shown with your answers to anyone, admins included, so your CTO sees "the team rated Copilot 4.1/5" and never "Sarah rated Copilot 2/5, and wrote 'honestly I prefer Claude'."
Why does this belong in a methodology post? Because the quality of the data depends on it. People answer candidly when they know who will see what they say, so we tell them before they start, and we tell them the limit too: in a small group, or from something they write in a comment, someone may still be able to work out an answer is theirs. We've watched engagement surveys lose their candid answers for twenty years once people stopped believing the promise. We would rather say where the limit is.
A worked exampleLink to this section
A 120-person firm pays for 80 seats of an AI assistant at A$60 a seat a month. Its blended hourly rate is A$110.
Over 8 weeks, 74 of those 80 people answer at least one pulse (a response rate of 92.5%). Their answers work out at an average of 2.7 hours saved per person per week.
Do the maths:
- Hours saved in a year: 74 people × 2.7 hours × 48 weeks ≈ 9,590 hours
- Their value at A$110 an hour: about A$1.05 million
- What the tool costs in a year: 80 × A$60 × 12 = A$57,600
- Ratio: roughly 18× return
That ratio is an estimate, and every part of it is shown, so you can test any of them before it goes into a board paper. It's not a number we ask you to take on faith.
Where you should push backLink to this section
Three limits worth knowing:
- Self-reported time is self-reported. People overestimate savings on tasks they enjoy and underestimate on tasks they dread. The 2.5× cap bounds how far one person's answers are scaled up; nothing fully fixes the rest. Pair our numbers with your own judgement.
- Non-responders are a real problem. The higher the response rate, the more the figure describes the whole team. At 50%, the half who didn't answer may be the people struggling the most. We show the response rate beside every figure so you can't miss this.
- ROI is a proxy, not the goal. Sometimes a tool costs more than it saves, and you keep it because it opens up something you couldn't do before. The GAiGE tells you the numbers; you decide what the numbers mean.
In shortLink to this section
Cap the extrapolation. Use your numbers, not ours. Always show response rate. On an anonymous pulse, show a figure only once five people have answered. Be open about the limits, and give the reader a number they can defend in six months' time.
If that sounds boring, it is, slightly. The ROI claims that hold up in front of a CFO tend to look like this.
How the figures are worked out in the product: Understanding ROI and How we work out hours saved. Questions about the methodology? Talk to us.
Colin Cardwell
Founder of AiGILE and The GAiGE. Colin has spent thirty years designing custom software for tough business problems.
Connect on LinkedIn (opens in a new tab)