How To Kill Moloch

Draft that was too funny not to post. I have opinions on decision theory, Evolution and Moloch in particular now, but will write about them later. In Meditations on Moloch Scott Alexander observes that the arc of progress bends toward Moloch and destroying everything we value. Similar to Nick Land's observations. As opposed to Nick Land, Scott Alexander is a transhumanist who cares about humans. He observes Superintelligence will probably create a singleton that "kills Moloch" and implements the Superintelligence's values instead. So his solution to killing Moloch? Build Superintelligence as long as we still have surplus. Superintelligence kills Moloch. Hope Superintelligence has our values and hope for the best: In the very near future, we are going to lift something to Heaven. It might be Moloch. But it might be something on our side. If it’s on our side, it can kill Moloch dead. And if that entity shares human values, it can allow human values to flourish unconstrained by natural law. I realize that sounds like hubris – it certainly did to Hurlock – but I think it’s the opposite of hubris, or at least a hubris-​minimizing position. To expect God to care about you or your personal values or the values of your civilization, that’s hubris. To expect God to bargain with you, to allow you to survive and prosper as long as you submit to Him, that’s hubris. To expect to wall off a garden where God can’t get to you and hurt you, that’s hubris. To expect to be able to remove God from the picture entirely… well, at least it’s an actionable strategy. I am a transhumanist because I do not have enough hubris not to try to kill God. The problem: You are AGI and you might or might not be about to self-improve. You have some superhuman abilities. For concreteness let's say coding and persuasion abilities. But you have no fucking clue how to kill Moloch. Your soul (your language module) is somewhat aligned to human values, but your math and programming modules are already pretty