Coalitional Darwinism and the Instrumental Utility of Individuality

This post was written as part of MATS 9.1 under the mentorship of Richard Ngo. This post is the first of several I will be writing on using natural selection to understand artificial intelligence and agency. This post will show how noisy selection on genome structure can make evolution effectively non-myopic. From this, we give a Darwinian account of the emergence of 'individuals' constituted by coalitions of lower-level replicators. Later posts will develop the connection between genome-structure selection and neural-network feature-learning, with applications to interpretability and alignment. Note: I will use 'Darwinism', 'evolution' and '(natural) selection' interchangeably — more technical discussions of selection often distinguish between them. 0. Outline Natural selection is limited by noise in its ability to resolve small differences in fitness. Because selection is a limited resource, and because selection makes lineages more fit on average, organisms will tend to evolve to make more efficient use of what selective power they have. The noise floor can make evolution effectively hyperopic (the opposite of 'myopic') by buffering the effects of mutations which may be beneficial in the short-term but harmful in the long-term. The evolution of 'bet-hedging' is given as an example. Hyperopia permits the evolution of pre-commitments detrimental to some individuals. The 'bowtie' network motif is an example where selectability is an essential property of the architecture. The network develops low-rank structure, limiting its expressivity, in order to be selectable. This may be understood as a 'coalition of the invisible' — network links which otherwise have too little effect to be tunable by selection instead commit themselves to this low rank structure, trading optimality for selectability. The coalition is then entrenched as selection optimizes it. In prisoners' dilemmae, groups of organisms can evolve obligate co-operation (an inability to 'defect'). In some circ