Why we built The GAiGE — and what most companies get wrong about measuring AI ROI

Three years ago, a customer asked us how to measure whether their AI tools were actually working. We had a good answer for the nine other steps in our adoption guide. That tenth one — measurement — we didn't have a good answer for at all.

When we started AiGILE, we built a ten-step guide to AI adoption for mid-sized businesses. Step one was pick the right problems to attack. Step two was pick tools that fit those problems. And so on down to step ten, which was some variation of measure the effectiveness of the tools you're using. It was the obvious final step. No one finishes an adoption plan without it.

It was also the step that kept embarrassing us. Customers would nod politely through the first nine steps and then ask, at the end, "OK — so how do we actually measure this?" — and we didn't have a good answer. Vendor dashboards didn't exist yet in any meaningful form. Our fallback was "send a survey". Our customers dutifully sent surveys. The response rates were dismal, the results were thin, and nobody learned very much.

That wasn't enough to stop the AI hype. Adoption kept accelerating — wildly, in some places — in a way that was mostly unplanned, mostly uncoordinated, mostly invisible to the people paying for it. Especially at mid-sized firms without a dedicated AI officer to care about the question. Tools got bought, rolled out, used, ignored, renewed. Nobody really knew what any of it was doing.

We realised, somewhere in the middle of that, that the problem wasn't going to measure itself.

What most teams settle for — and why it's not enough

By year two, vendors had started publishing basic usage telemetry. Seats, sessions, prompt counts. That was genuine progress over having nothing, and we saw CTOs gratefully latch onto it.

Usage data is the most common measurement mistake we see now. Not because it's wrong — it isn't — but because it's so obviously insufficient that it's surprising how many leadership teams treat it as the whole answer. If you only know how often a tool gets opened, you don't know how your team is using it. You don't know what they're using it for, or how well it's going, or whether they'd recommend it to a colleague. You don't know why three people on the same team have quietly stopped using it. And you don't get anywhere near a defensible ROI number.

Usage tells you someone logged in. It doesn't tell you whether any of it mattered. A CTO who closes the laptop confident because *"usage is up 40% quarter on quarter"* is, in our experience, exactly the CTO who'll be surprised when the board asks about business impact.

The Chrome extension idea — and why it changed things

We knew asking people was the right direction. The catch was always response rate. An email survey is, quite literally, the definition of friction — it's in the way, it arrives at the wrong moment, it costs you ten minutes, and the reward for finishing is that you've filled in a form. Response rates of 20-40% are normal. We needed something radically different.

About a year ago we started playing with a different idea: a Chrome extension that delivered a short prompt in the moment someone used a tool. Right after the session, not weeks later. Two or three questions, not twenty. A few seconds, not minutes. The response rates would go up because the friction would go down.

There's a personal angle there too — before AiGILE, a lot of my background was in game design and gamification. Friction and reward are the two levers game designers think about every day. A badly-designed survey is all friction, no reward. A well-designed one is quick, specific, and slightly satisfying to complete. That's not cosmetic; it's the difference between a 35% response rate and an 85% one.

We started calling them Pulses instead of surveys. Not because "pulse" is cuter, but because the product needed to feel different — a brief check-in rather than a corporate form. The name set the design brief.

The methodology we didn't know we needed

Here's the admission. Our original plan was to build the extension, aggregate some numbers, and worry about the methodology behind "ROI" later. Good enough, ship it, we'd refine.

The tool we were building The GAiGE with — Claude — pushed back, repeatedly, every time we reached a shortcut. It kept asking awkward questions. "What happens when one user reports 20 hours saved and the rest report 1? Do you really want to extrapolate that? What about response-rate bias? What are you capping at?"

Every one of those was a question we'd planned to deal with after launch. Every one of those, it turned out, would have materially embarrassed us if a real customer had asked it first. So we leaned in, talked it through, argued with it, tested it, and ended up with a methodology we can publish — the 2.5× extrapolation cap, the response-rate-aware reporting, the aggregate-only privacy posture, the normalised health scores on every gauge. That work, more than any other single thing, is what separates The GAiGE from a slick survey tool. It's also the work I didn't plan to do.

I wanted to note that specifically because we're living in an era where "built with AI" is usually a boast about speed. For us it's also a genuine quality story. The product is more rigorous because we had a collaborator who wouldn't let us off the hook on the hard questions.

Why us, honestly

A fair question to ask: with AI-assisted development as cheap and fast as it now is, why does AiGILE specifically get to build this, rather than any of the dozens of teams who could spin up a similar product in a weekend?

Three years of sitting inside mid-sized businesses trying to get them to adopt AI gave us a set of intuitions we didn't know we had. We know which metrics executives actually trust in a board pack and which ones they'll ignore. We know how teams actually use these tools (not how the vendor demo says they'll use them). We know which questions cause defensive behaviour if asked the wrong way, and which ones unlock honest feedback. That's not consultant polish; it's accumulated pattern recognition.

My game-design background matters too. A weekend's worth of vibe-coded MVP can produce a working survey tool. It's unlikely to produce a survey tool with a 75%+ response rate. That gap is where we've spent most of our time.

Where we are, and what we'd like you to do

It's early days for The GAiGE. We've made a lot of decisions we believe in and we've published the methodology so you can push back on any of them. We also know there's plenty we haven't built yet, and plenty of signal we haven't yet figured out how to surface well.

We'd genuinely like you to join us on that journey. The trial is on us, no credit card — long enough to install the extension, roll it out to your team, and see whether the dashboard tells you something you didn't already know. If it does, you'll find the pricing pleasantly unexpected: we've fully internalised how AI is changing the economics of SaaS, and what you'll pay is a flat fee for your whole organisation, not per-user. We think that's where this category is going, and we'd rather lead than chase.

Three years after that first embarrassed moment with a customer asking "how do we measure this?" — we've finally got a good answer. Come try it.