Getting past the pilot: Why so many AI test projects have trouble scaling

It’s an increasingly common tale within corporations today: The AI project performs admirably in testing during the pilot phase, gets the green light for a broader rollout…and then stops working properly; Or it fails to deliver the expected business results. Finger pointing, recriminations, and embarrassment ensue. The problem is not always the technology. In fact, the fault is often in the planning, processes, and expectations that companies have established—or not established—around their AI projects, according to business leaders who spoke at a roundtable discussion at Fortune Brainstorm Tech this month. For starters, not every AI project deserves to be rolled out widely, said Amgen Chief Technology Officer Sean Bruich. “It’s so easy with a pilot to let a thousand flowers bloom,” he said. That’s not a bad thing, since it encourages experimentation. But, he said, “the key to making pilots scale successfully is actually having a wide number of ideas, but a very tight governance on which pilots are actually greenlit.” A key criteria before taking the next step, said Salesforce Chief Customer and Commercial Officer Lashonda Anderson-Williams, is understanding the intended outcome of the project. Too many companies are focused on the successful implementation of AI features—the technological bells of whistles—instead of the business outcome, she says. That mentality is a recipe for disappointment: The AI features work great, but the new technology isn’t driving meaningful business results. Agents needs a map When it comes to agentic AI, Anderson-Williams noted, a detailed understanding of the workflow—which individuals, groups, or touch points are necessary to complete a task— is critical. What a lot of companies are finding, she said, is that documentation of the workflow either doesn’t exist or is poorly documented: “When you put AI on top of that, the expectation is you’re going to see some magic, and there’s no magic there.” Access to data is a particularly