[New Paper] Prioritizing Risks from AI: A Delphi Study of 272 Experts

TL;DR: We ran a Delphi study with 272 international AI experts to prioritize 24 AI risk domains from the MIT AI Risk Domain Taxonomy. In a business-as-usual scenario, experts judged a more than 10% chance of catastrophic outcomes (i.e., ‘more than 1 million human deaths or more than a USD 100B in financial loss or civilizational-scale intangible impacts’) from 18 of our 24 AI risk domains over the next five years. They also identified a responsibility gap: AI users and affected stakeholders are most vulnerable, while general-purpose AI developers and governance actors are seen as most responsible for reducing the risks. Below are three of the key findings and related visualizations. Key finding 1: Experts judge that many risks could cause catastrophic outcomes under current trajectories18 of 24 risks were judged to have a more than a 10% chance of causing catastrophic outcomes (which could include more than one million deaths, more than $100 billion in financial losses, or other harms) by 2030 under a business as usual scenario.Figure: Experts’ mean catastrophic risk probability under business as usual and with pragmatic mitigations. Note: “Business as usual” assumes organizations and governments continue their existing practices but do not implement additional AI-specific risk mitigations; “Pragmatic Mitigations” assumes organizations and governments make pragmatic, cost-effective efforts to address AI risks.Key finding 2: Those most vulnerable to AI risks are not those most responsible for addressing themAccording to experts, general-purpose AI developers and governance actors such as governments, regulators, and standards bodies hold primary responsibility for addressing AI risks. In contrast, AI system users and affected stakeholders such as members of the public are most vulnerable to AI risks.This mismatch means that those who are most responsible for addressing AI risks are not those who are most vulnerable, leading to misaligned incentives in addressing the