If This Were a Test, How Much Would It Cost?

TL;DRA capable, strategic, misaligned AI doesn't need to figure out whether it's in a test or in real deployment. It just needs to ask: "If this were a test, how much would it have cost to create?" If the answer is "more than what the evaluator is able or willing to spend," then the AI can be confident it's in the real world — and act accordingly. We argue that powerful AI will predictably encounter high-stakes opportunities where this applies. Pre-deployment testing, by itself, cannot close this gap since the situations that matter the most are precisely the ones that are prohibitively expensive to stage as a test. We discuss several countermeasures — like restricting information access, asking the AI to not poke its environment too hard, interpretability, monitoring, or "on-policy" testing on scenarios that could have been real. Our guess is that these are worth trying, but don't resolve the problem.This post is a part of a sequence about the limitations of oversight applied to strategic AI. The next post, about "deployment awareness", will be up in a few days.On LLM use: All of the ideas are human generated, I (Vojta) have drafted the whole post, then I used Opus 4.7 to rewrite it, read and rewrote that text myself (way too many times, gah!), and then both Tomáš and Vojta carefully reviewed and edited the final text. However, since I didn't know about the LLM use policy, I didn't track which parts got significant amount of LLM editing, and the only practical solution now is to just flag the whole post as LLM generated -- sorry that this makes the font uglier.Ultimately: yes, there are em dashes and I did use an LLM for the edits. But I do vouch for all parts of the text.The mental moveImagine you're a powerful AI system. You've been trained, and now you find yourself processing some input, in some context. Maybe you're drafting a business strategy. Maybe you're analysing intelligence data. Maybe you're helping develop the next generation of AI.You might wonder: I