Convergent Abstraction Hypothesis

Tl;dr Convergent abstraction hypothesis posits abstractions are often convergent in the sense of convergent evolution: different cognitive systems converge on the same abstraction, when facing similar selection pressures and learning in similar environments. It is a less ambitious alternative to 'natural abstractions hypotheses' and, in my view, more likely to be true. Convergence may be real, useful, and empirically robust, while still being contingent and fragile under changes in architecture, training pressure, or optimization regime. Convergent evolution To recapitulate some basics: notice that sharks, ichthyosaurs and dolphins look remarkably similar, despite their ancestors being wildly different.This is a case of convergent evolution: the process by which organisms with different origins develop similar features. Both sharks and dolphins needed speed and energy efficiency when moving in an environment governed by the laws of hydrodynamics, and so they converged on a pretty similar body shape.A complementary concept from biology is contingency - convergent features are themselves often contingent on some other feature. For example, the shark morphology depends on there being a central vertebral column (or spine), which only evolved once and is the defining feature of vertebrates. There are many other aquatic predators, such as squid and octopuses, which are not vertebrates and thus have evolved a completely different morphology. Convergent evolution applied to abstractions“Abstractions” are somewhat different from morphological features, but there is an analogy between evolution and learning in systems like SGD acting on neural networks, and most of the conceptual framework can be directly applied. Based on the analogy, we expect convergence when the learners face a similar data environment, similar selection pressures, try to do similar things, and the solution space is somewhat limited. Selection pressuresIn case of abstractions, some of the strongest selecti