Could AI Outgrow Consciousness?

TL;DR: Conditional on believing in illusionist theories of consciousness, it seems plausible that AI systems will be conscious above some threshold of complexity but surpass a need for consciousness once they become sufficiently advanced.Consciousness for humans is confusing enough as it is but adding this onto a discussion of conscious machines makes things even more complicated. Some people are pretty confident that AI systems will not be conscious because there is something intrinsic about biology. Others note that the possibility is more likely than we might otherwise think. We think a neglected question to consider is whether AIs above a certain threshold of complexity may need consciousness but then no longer require it above some other threshold. To understand this take, though, it is important to unpick some of the concepts here.What is consciousness?Consciousness is clearly a bit difficult to pin down, with the most popular definition just saying it is where there is something it is like to be an individual. This doesn’t really clarify things! So it is worth thinking this through some more.We would contend a conscious experience is one which is necessarily subjective. We might look at a brain scanner to see the parts of a brain that light up when someone is happy but we do not directly observe the subjective nature of that experience. A doctor suffering from congenital insensitivity to pain may be able to describe exactly when, where, and (from a medical perspective) why her patient feels pain, but would find descriptions of the sensation unrelatable and inaccessible. A conscious experience need not amount to pleasure or pain. There are other experiences that are pretty indifferent, for example the experience of seeing the colour red. Cognition, by contrast, is much more narrow. This is where a stimulus incurs a response such that an entity “thinks” in a very narrow way. To see the difference, it is worth considering an entity eating food. The conscious exp