Claude is becoming more agentic. Amanda Askell is thinking through what that means

Amanda Askell spends her days thinking about how to ensure Claude, Anthropic’s AI chatbot, operates with a sense of morality. As AI models move from chatbots toward agents that can complete tasks on their own, the decisions these models make stand to become far more consequential. Askell, a member of the technical staff at Anthropic, sits at the center of the company’s effort to give Claude an ethical compass, a responsibility that grows as the system’s capabilities expand. “As models are more autonomous and take actions over longer horizons, suddenly they have a lot more decision points that you have to map out and make work well in advance,” she tells Fast Company. There is a clear difference between asking a large language model to discuss the morality of buying stock in a defense company and asking it to manage a user’s investment portfolio without day-to-day human input. Askell says part of the solution is encouraging Claude to be responsive and, like a friend, to understand a user’s values without imposing its own idiosyncratic ethics. Today, Anthropic communicates its values through a written and evolving constitution, which outlines principles such as safety and helpfulness, along with guidance for resolving conflicts between them. As AI becomes more capable, that document could expand to cover new scenarios, Askell says. Or it could shrink, as Claude develops more expertise in navigating complex situations. The agentic era is also changing Askell’s own work. She uses Claude often, including to red team her ideas and identify edge cases. “My standard right now is, don’t treat Claude as more reliable than a human personal assistant,” she says. The following expanded conversation, part of Fast Company’s AI 20 package, has been edited for length and clarity. Right now, we’re used to interacting with models in this digital text environment. You can ask them a question like: Is it ethical for me to invest in this defense contractor or invest in