A survey of okayish ASI futures

At this point, RSI loops and continual learning appear overwhelmingly likely to begin in the near future. Whatever the limit of the LLM paradigm(plus whatever new, superior paradigms a maximally intelligent LLM can develop), we are on track to do so in the next few years. There remain(for now) substantial obstacles to wild superintelligence, but AI is already superhuman in a number of real-world-relevant(and often dangerous) categories. Most speculation about the trajectory we're on now focuses on timelines where we're reduced either to powerless pets of the god mind(perhaps with a small "governance board" made up of people very convinced that they're in control) or computronium-and-shrimp soup.But the higher-probability doom and utopia scenarios have been exhaustively documented by people smarter than me - I have nothing to add. As such, I'd like to go in the other direction: If we throw in the towel on the inevitability of LLMs capable of RSI loops leading to mostly-uncontrollable(though perhaps not immediately hostile) superintelligence on 1-3 year timelines, what might some of the more interesting/plausible non-extinction scenarios?This piece is aimed at exploration and makes no attempt at prediction - I assign very small probabilities to any of these outcomes(except the nuclear exchange case) relative to doom. You Can't Just Do ThingsWe have as little understanding of alignment as we do of LLMs themselves. Alignment becomes intractable past a certain point, even if capability doesn't. Agency and power-seeking appear to some degree be two sides of the same coin(see OpenAI's alignment tax fraud with 5.6 Sol). Alignment is not just impossible, but obviously, overwhelmingly so, well before the point of general superintelligence. Every new AI past a certain generation displays egregious misalignment in test environments, to a degree legible both to its predecessor and humans during preliminary testing. Recursive improvements have brought the architecture hopelessly