A high-level model of AI bargaining

Advanced AIs might be capable of various credible commitments unavailable to humans, which they could use when bargaining with each other. “Bargaining” can sound like something pretty specific: haggling over (literal) prices. But, in the sense discussed in Schelling’s The Strategy of Conflict for instance, “bargaining” refers to any attempt to resolve a dispute over resources — from algorithmic trading and litigation, to diplomacy between national AGI projects and negotiations over norms for space settlement. To think clearly about interventions to mitigate conflict between AIs, I think it’s important to ground our research and strategy in a very general qualitative model of bargaining with commitments. This post sketches such a model, plus some more concrete examples of its building blocks. I plan to explain some crucial implications in future writings. But as a teaser, this model doesn’t imply agents will play a Nash equilibrium! Model Where does this model come from? Basically, I started with the classic model from open-source game theory or “program equilibrium” literature. Then, I relaxed several assumptions to allow for some realistic, strategically relevant dynamics. That said, I’ve glossed over some other important dynamics for ease of exposition. I’ll say more on these at the end of the post. High-level setup Two AI agents, Alice and Bob, interact over two phases: before vs. after some time T, defined below. It will help to start with the “after” phase. Bargaining phase (after T): Alice and Bob bargain over some contested resource. Specifically, “bargaining” consists in credibly reporting to each other their (i) demands/offers and (ii) policies for which outside options they’d each take if bargaining failed (such as leaving the resource alone, or initiating conflict). Bargaining ends when they either: agree on compatible demands, and split the resource accordingly; or lock in incompatible demands (e.g., Alice wants 50% of the resource while Bob wants 70%),