Our fears about AI are really fears about capitalism

As writer Ted Chiang has observed, “Most fears about A.I. are best understood as fears about capitalism.” When we imagine rogue AI systems optimizing the world to death, we’re actually describing what many corporations already do. If an organization’s ethos is misaligned, any technology platform it creates will amplify that misalignment. Take a moment to reflect on the organization you hold most dear: Does it have an ethos, or is it drifting without one? Does it fight for human flourishing or against it? Is it creating value or extracting it? Are you shaping its ethos, or is it shaping yours? No matter your formal title, you have influence over this organization. Yet if you cannot answer these questions with confidence, you are not in control. You are a passenger in a vehicle you think you’re driving. But if nobody is consciously driving, where is the organization going? If the behavior of superorganisms were random, we would see as many evolving toward flourishing as toward corruption. Instead, the vast majority wind up in the same place, with indistinguishable values. I recently had a conversation with the CEO of a multibillion‑dollar company who’d come to see me for help with a baffling problem. He had made a bold new bet on the promise of AI to revolutionize his business. He’d personally overseen the development of a new AI product and tested it with customers. They loved it. Satisfied that he had created something amazing, he turned it over to the rest of his team to commercialize and scale. Six months later, this new product was going nowhere. The sales team wasn’t selling it. Developers were loath to work on it. Marketing would not promote it. When confronted, the executives would insist that the product was part of their quarterly priorities. Each of them was telling their teams to focus on it. Yet, somehow, line employees and middle managers were consistently avoiding the AI product. It was a pattern throughout the organization. When the CEO personally inte