AI as a Social Technology, by Henry Farell

Last week, I attended a talk ‘AI as a social technology’ by Henry Farell (HF) at the Blavatnik School of Government in Oxford. In this post, I list various thoughts or recollections I have. If I had more time, I would create a more coherent flowing narrative, but I’d rather get something out than nothing.Caveats:These notes are based on my memory of the talk, so high chance I am mis-representing some of HF's views.I wrote this before seeing this article 'Large AI models are cultural and social technologies' by HF et. al., written in March 2025. Based on a quick skim, seems their views have not changed significantly in the past year.HF’s key beliefsAI is already a humungous deal and society will change a lot.AI discourse needs more input from social scientists and collaborations between them and STEM people. The discourse is currently dominated by computer scientists. In particular, the two dominant narratives represented by AI 2027 and AI as a normal technology are too simplistic and under-estimate how weird and unpredictable the consequences of AI will be.These computer scientists are overly influenced by the sci-fi that they (and HF) have read, and their writing is like reading bad sci-fi. “We should leave the sci-fi to sci-fi authors.”HF proposes thinking of AI as a ‘Lossy Information Aggregation Tool’ [terminology mine]. Modern AI systems have aggregated huge amounts of information and data and present users with a lossy representation of all that information. Historical examples of this which had huge consequences are state bureaucracies (see Seeing like a state by James Scott) or markets (see Hayak’s ideas on markets and prices as ways of aggregating and distributing information). The fact that these tools for aggregating information are lossy are crucial to understanding their limitations and impacts.AGI is unlikely to happen soon. Thinking about what would happen if AGI was created is only useful as a ‘thought experiment’. Instead focus should be put on what