Quantitative AI risk assessment: a starting point

Current AI risk management relies on qualitative approaches, much like nuclear safety before 1975. We propose a shift to quantitative risk modeling, following the approach that transformed nuclear safety. We propose a methodology and demonstrate it by building nine probabilistic models of AI-enabled cyber attacks. This is a first attempt at AI risk quantification. We invite criticism and hope this can be a starting point for the kind of iterative improvement that made nuclear safety robust.Introduction — why quantitative risk modeling mattersIn 1975, the Nuclear Regulatory Commission (NRC) published WASH-1400, the first systematic application of probabilistic methods to nuclear reactor safety. Before this, nuclear reactor safety was grounded in a conservative, qualitative approach. The argument was that consequences of even the worst accident were effectively zero because physical barriers would prevent any radioactive release. But in 1966, experts recognised that a core meltdown in large reactors could breach those barriers. Regulators now had to argue that severe accidents were possible but sufficiently improbable. That required specific probability estimates.The shift to quantitative methods was not straightforward. When WASH-1400 began, one technical advisor wondered, "Do we dare undertake such a study till we really know how?" The 1975 report drew intense criticism, and the NRC partially rejected its findings in 1979. Yet, even imperfect quantification brought three benefits, which compounded over time.First, quantification made safety claims auditable. Because assumptions were now explicit, reviewers could critique specific data sources, modeling choices, and uncertainty estimates. As Keller and Modarres note, probabilistic risk assessment (PRA) forced nuclear plant operators to "write down all of the assumptions involved in reactor operation," assumptions that had previously remained implicit. This invited specific disagreements and enabled iterative improvem