How valuable are weak AI safety regulations?

To prevent superintelligent AI from killing everyone, I would like there to be a strong international agreement banning the development of ASI until it can be proven safe. But that sort of agreement requires a lot of political buy-in and coordination. In the meantime, it may be easier to get light-touch AI safety regulations passed. To what extent do weak regulations decrease extinction risk? In this post: Part I discusses routes by which weak regulations can reduce extinction risk. [More] Part II considers some downsides of weak regulations. [More] Part III reviews specific categories of weak regulation and how they might reduce risk. [More] Cross-posted from my website. I. Ways weak regulations can reduce risk Directly reduce extinction risk Weak regulations can't do much to decrease misalignment risk, but they can have small effects at the margin. GPU tariffs or moderate restrictions on GPU exports slow down AI development in other countries, and reduce competitive pressure to some extent. [1] Mandatory safety testing has some small chance of catching catastrophic issues before they happen. [2] Empower future efforts to reduce extinction risk What I really want is a global ban on superintelligent AI until it can be proven safe. To get that, we will need some regulations along the way. For example, regulators will need to know who has the ability to develop advanced AI systems, which means we need some sort of monitoring of AI developers or AI hardware. Reveal warning shots At some point before AI kills everyone, it might do something scary enough to trigger governments to pause AI. [3] Weak regulations can make warning shots more apparent. If AI companies are required to publish safety tests, and there are legally mandated whistleblower protections, then it's more likely that scary AI behaviors will come to light. Shift the Overton window Passing weak regulations in the near future may make politicians more amenable to strong regulations later on. (I say "politic