Pulling on AI Safety (with money)
This was my submission to Dwarkesh's essay contest, which had an 1000 word limit. If you want a longer version, or have specific questions, please comment. I answered the question "If you were in charge of the Open AI Foundation right now, what exactly would you do?"I'm unsure how good marginal push funding in the current AI safety ecosystem is, which is obviously a fundamental consideration when deciding to put money into pull funding. Still, the fact that I have seen practically nothing about this topic makes me think it probably deserves far more discussion than it's received.We need a way to turn money into effective institutions. If only we knew of some–The Profit IncentiveThe vast majority of philanthropic money is in push funding, supporting people and organizations that already do good work. In AIS, this happens via salaried positions in frontier labs, as well as grantmaking in academia and the non profit sector.Pull funding funds outcomes rather than processes. It’s best suited to social goods which do not generate profit, and cases where the desired outcome is clear but the path forward is not. There are many cases like this in AI safety; I do not know which researchers or organizations I would fund in order to find a cheap security intervention which reduces model-weight exfiltration risk by some percent according to independent superforecasters. I do know that that would be good. Pull mechanisms of the past have either been limited in funding (<$200m), like DARPA's, or targeted massive pre-existing industries, like the Pneumococcal AMC. What would happen if billions of dollars were committed to novel AIS goals? Investors could fund promising contestants in exchange for equity, just as they do with other high upside companies. For-profit funders one shot Nan’s qualifications: they resource generously because it will make them money. They give leeway because it will make them money. They invest in whatever problems make money, which AIS experts in turn deci