Enterprises spent years chasing efficiency gains from AI. Industry leaders now say that this metric isn’t the best to measure the success of AI. Here’s why the smartest marketers are abandoning the speed scoreboard for one that actually tracks growth.
Ninety-five percent of enterprise AI initiatives never make it to full-scale production. Not because the technology fails, but because companies spent years chasing the wrong scoreboard.
When LLMs first arrived, enterprises scrambled to implement them, training staff and racing toward the cost and time savings that frontier, and smaller AI vendors, promised. Efficiency became the metric that mattered: could you get the same output, or better, with less staff time and less spend?
It worked, in a narrow sense. But companies soon realised it didn’t move the top line. Marketers are now realising that saving time on drafting emails or answering tickets was never going to be the thing that grew their bottom line — and that the businesses treating those savings as the finish line are the ones falling behind.
The Efficiency Trap
Despite its obvious appeal, using efficiency gains as the primary benchmark for AI success – such as drafting emails, responding to tickets, or going to market quicker – doesn’t necessarily mean the quality of outcomes increases.
Prioritising speed over outcomes can lead to lower-quality work, as employees learn to trust AI tools and deprioritise the attention to detail associated with more traditional ways of working. This was particularly true when GenAI tools first launched, when simply “using AI” for anything became a marker of success across many businesses.
“Our biggest clients treat production savings as the floor, reinvesting them to fuel the outcomes that matter: brand growth, market share, revenue,” says Tobias Cummins, Chief Operating Officer at AI marketing platform Pencil. “Delivering efficiencies will get you liked. Growth tells you you’re winning. The brands getting AI right are measuring what it grows, not what it saves.”
Many of the time- and cost-saving tasks easily automated by AI are also likely to be automated by competitors. So sacrificing creativity or quality for the sake of efficiency can seriously damage differentiation and competitive advantage.
According to Cummins, this is becoming increasingly important as brands and agencies advance in building their agentic workflows. He continues: “Prioritising effectiveness, on the other hand, often leads to the orchestration of multiple AI systems and tools. As companies shift towards agentic workflows, measuring AI performance based on strategic business outcomes has become more common.”
Effectiveness Has an Attribution Problem
But moving past efficiency doesn’t solve the underlying measurement problem — the difficulty of knowing whether AI is actually working. Time and cost are easy to track, which is why it’s been a tempting yardstick. Effectiveness is much harder to track, as it’s varied and different for each organisation.
“Outcomes are hard to attribute cleanly,” says Liz Duff, Co-Managing Director at independent agency Mediaplus. Duff argues that the success markers marketers have always relied on – not speed – remain the better measure of growth.
“Where AI genuinely reduces production costs, we’d rather pass those savings on,” she says. “The real test of effectiveness isn’t output volume, it’s whether campaign performance improves, whether clients see it in results rather than reporting, and whether that shows up in renewed and expanded relationships.”
Working Backwards
Marketers are starting to recognise that speeding up the time it takes to achieve an outcome isn’t progress if it leads to poor decisions, worse employee job satisfaction or consumer engagement. So working backwards from these goals – increased return-on-investment, employee satisfaction, or more engagement – which are all historic success metrics anyway, has proven to be a more effective and logical approach for many marketers.
“We ended up seeing scoreboards full of “green” on making tasks more efficient, but couldn’t connect them to output or outcomes,” says Roy Armale, DEPT’s Chief Product Officer and the mind behind WPP’s AI orchestration platform WPP Open.
Armale takes the approach that even focusing too much on effectiveness as an AI success metric misses the mark.
“We end up in the same trap as before: shoehorning intermediary metrics like “feedback” and “quality” because effectiveness, like revenue growth, is hard to track and correlate.” Armale believes that working backwards from outcomes helps marketers understand where AI can drive the most benefits.
“We need to follow the data,” he explains. “MIT Nanda’s research shows that 95 percent of enterprise AI initiatives never reach full-scale production because of focus on broad benefits instead of working backwards from outcomes into deliverables, processes that enable those deliverables, and then use AI to optimise those processes.” With this approach, Armale outlines, companies can track both efficiency and effectiveness, but both with a view to enable better outcomes.
