How Amanda Askell is teaching Claude to make ethical decisions

As AI models move into a more agentic era, shifting from chat to completing tasks, they’ll be making real and increasingly consequential decisions. Whether they bring any sense of morality to those decisions is the kind of problem Amanda Askell is exploring. A philosophy Ph D from New York University, Askell spent two years at Open AI before joining Anthropic in 2021, where she sits at the center of the company’s effort to instill in Claude an instinct for ethics—a responsibility that grows as the system’s capabilities expand. “As models are more autonomous and take actions over longer horizons, they have a lot more decision points that you have to map out and make work well in advance,” she says. There is a clear difference between asking a large language model to discuss the morality of buying stock in a defense company and asking it to manage a user’s investment portfolio without day-to-day human input. The latter raises a harder problem: how AI should make value-laden decisions on a human’s behalf. Askell says part of the solution is encouraging Claude to be responsive and to understand a user’s values without imposing its own “idiosyncratic ethics.” Today, Anthropic communicates those values through a written and evolving constitution—an effort led by Askell and formulated as instructions to Claude—that outlines principles such as safety and helpfulness, along with guidance for resolving conflicts between them. As AI becomes more capable, that document could expand to cover new scenarios, Askell says. Or it could shrink as Claude develops more expertise in navigating complex situations. In the meantime, the agentic era is transforming Askell’s own role. She uses the technology often, including to test her work and identify edge cases she ought to consider. “My standard right now is don’t treat Claude as more reliable than a human personal assistant,” she says. “I also don’t treat Claude as much less reliable than that either.” This profile is part of Fast Company