How to run a company when the AI agents vastly outnumber the humans

Having a “human in the loop” is the typical recommendation for organizations using artificial intelligence for jobs in which there’s no margin for error. But what happens when it’s just not possible to keep a human in the loop?As the use of AI ramps up within organizations and as businesses delegate more tasks to agentic AI, it’s a challenge that many corporate leaders are starting to grapple with. “We’re seeing a material increase in the speed with which we can create things,” Zach Maybury, chief technology officer at online sports betting platform DraftKings said during a panel discussion at Fortune’s flagship technology conference, Brainstorm Tech, this month. Maybury said his company already deals with trillions of transactions and highly distributed workloads. Once you introduce agentic AI into the mix—with AI agents talking directly to other AI agents—the volume and complexity of the operation is far too vast for traditional approaches. “I can’t insert humans into those loops,” Maybury said. “We will never have enough humans to insert in all those loops.” Maybury was just one of several business leaders at Brainstorm Tech who discussed the challenges of managing AI in mission-critical situations. “In high-stakes environments like health care, it’s not taking the wrong tee-shirt size if you’re a retailer, it’s a life on the other side of it,” said Salesforce chief customer and commercial officer LaShonda Anderson-Williams. While there is no one-size fits all solution to these challenges, many of the panelists described techniques and frameworks that have proven successful for them. Anderson-Williams said taking a clear-eyed look at AI use cases, and understanding what outcome you’re ultimately aiming for, is critical. Equally important is nailing down the proper governance framework—that is, a clear policy and set of rules that stipulate where and how the AI is allowed to operate, how it’s designed, and who is responsible for various parts of the process.