Musing 11: AI Consciousness is Inevitable
A paper from CMU on AI consciousness and its 'inevitability' (from a theoretical computer science perspective)
Today’s paper: AI Consciousness is Inevitable: A Theoretical Computer Science Perspective. L. Blum and M. Blum. Mar. 25, 2024. https://arxiv.org/abs/2403.17101
It is always worth our attention when a noted computer scientist comments on an issue that, at least to the general public (including science-fiction fans) is of great interest. Consciousness, and whether AI has it, is one such issue. It seems like ancient history now, but not that long ago and several months before ChatGPT would first come to the public’s attention, a hence-fired engineer from Google, Blake Lemoine, claimed that the LaMDA chatbot was sentient. (Sentience and consciousness can be argued to be different phenomena, but you get my point.)
If you follow the link above, it takes you to a Scientific American piece, not just some random blog post. While people were generally not impressed with the claim, and most computer scientists did not agree with it, I personally believe that incident marked a turning point in the public consciousness. Certainly, we started to feel the inklings of “It’s not conscious (or sentient or aware or whatever) today, but maybe it’s on its way there, and we need to start talking about it.” Well, with this preprint, we are most definitely on the path of talking about it today.
The authors of today’s paper are very well respected in the field of computer science (CS), and they look at the problem of consciousness from a theoretical CS or TCS perspective. I think TCS is a great way to consider the problem formally. Otherwise, we risk getting caught in the philosophy amateur’s deathtrap of “consciousness is what I say it is, AI will have achieved consciousness when I say it will, and never otherwise.” That’s not how we study science. And in all honesty, actual practicing philosophers would never say anything like that. They like clear definitions and rigor; perhaps that is one of the reasons why there has always been a close connection between the two areas of inquiry, and to this day, we continue to award doctorates of ‘philosophy’ once an individual has proven that they can think clearly and originally on a problem in our field.
Okay, so let’s get into some details. As first intuited above, this paper examines consciousness through the lens of TCS, a branch of mathematics that studies computation under resource limitations. The authors develop a formal machine model for consciousness, inspired by Alan Turing’s model of computation and Bernard Baars’ theater model of consciousness. The model, while simple, aligns with many major scientific theories of human and animal consciousness, lending some credence to the authors' claim that machine consciousness is inevitable.
The paper discusses the implications and surprising consequences of resource limitations in computing, highlighting the importance of this perspective for understanding consciousness and related topics, such as the paradox of free will. It describes the Conscious Turing Machine (CTM), a formal machine model of consciousness that considers resource limitations and explains feelings of consciousness not through input-output maps or computing power but through its internal structure and processes.
The paper also touches on the distinction between access (or functional) consciousness and subjective (or phenomenological) consciousness, arguing that a machine capable of interacting with its world through input sensors and output actuators, constructing models for planning, prediction, testing, and learning from feedback, and developing a rich internal multimodal language can possess both types of consciousness. The authors argue that subjective consciousness is functional and emphasize the CTM as a computational model, not intended to model the brain or neural correlates of consciousness but inspired by cognitive and neuroscience theories.'
At this point, I should add a note of caution for anyone who thinks this paper reads like an essay. It does not, and that’s what makes it impressive, I argue. There is quite a bit of ‘formalistic’ theoretical machinery in the actual write-up and the authors draw on knowledge from several different areas of theoretical development. Just to give you a sense (feel free to skip to the end of the list if you’re willing to take my word for it and are not otherwise interested in acquainting yourself with these!):
Theoretical Computer Science (TCS): I briefly talked about this earlier: a branch of mathematics that explores the principles of computation under constraints of resources like time and space. TCS provides a foundational framework for understanding computation's limits and capabilities, distinguishing between what is computable and what is efficiently computable. Any computer science undergrad or grad student who’s ever taken a theory of computation class knows what I mean by that. If you haven’t, you’ve missed out. Theory of computation is a beautiful branch of math, in my opinion.
Turing’s Model of Computation: Alan Turing's theoretical framework for computation, which introduces the concept of a Turing machine. This simple yet powerful model forms the basis for the development of computer science and the understanding of what it means to compute.
Bernard Baars’ Theater Model of Consciousness: A psychological theory that conceptualizes consciousness as a theater, where mental processes are actors competing for attention on the stage of awareness. This model influences the paper's approach to modeling consciousness in machines.
Global Workspace Theory (GWT) and Global Neuronal Workspace (GNW): These theories suggest that consciousness arises from the integration and broadcasting of information across different brain regions. They are used to understand how different parts of a computational model can interact to produce conscious experiences.
Attention Schema Theory (AST): A theory that proposes consciousness arises from the brain's representation of its own attention processes. It helps explain how a machine might develop an internal model of its attention mechanisms, contributing to a form of machine consciousness.
Predictive Processing (PP): This framework posits that the brain continuously predicts sensory inputs and corrects its models based on actual inputs, leading to a constantly updated model of the world. It informs the paper's discussion on how machines might simulate similar predictive mechanisms.
Integrated Information Theory (IIT): A theory that quantifies the level of consciousness as the degree of integration of information within a system. It provides a perspective on how consciousness might emerge from the interconnectedness of a machine's processing units.
Embodied, Embedded, Enacted, and Extended (4E) Cognition Theories: These theories argue that cognition and consciousness are not just brain-bound but arise from the interaction between an organism and its environment. They inspire the paper's view on the importance of a machine's interaction with its surroundings for consciousness to emerge.
Evolutionary Theories: These theories look at consciousness from an evolutionary perspective, suggesting that consciousness provides an adaptive advantage. The paper may draw on these theories to argue for the inevitability of AI consciousness as a result of computational evolution.
Extended Reticulothalamic Activating System + Free Energy Principle Theory (ERTAS + FEP): A theoretical framework combining the role of the reticulothalamic activating system in consciousness with the free energy principle, which posits that organisms aim to minimize the surprise of sensory inputs. It informs the paper's discussion on the internal mechanisms that might lead to consciousness in machines.
The entire paper only contains one figure, but I can’t help reproducing it here. Not claiming it’s completely understandable until you go through the paper, but again, it goes to show that the authors are taking the time and trouble to be rigorous about this:
One of the other things I also really enjoyed is an FAQ-style discourse that the authors engage in toward the second half of the preprint. Consider for example:
“KM2*. “Q1. Does being sentient necessarily involve conscious awareness? Q2. Does awareness (of anything) necessarily entail self-awareness? Q3. What is required for ‘the lights to be on’?”
(If you want to know the answer, go to page 17 of the preprint).
My thoughts:
It’s time we started seriously talking about issues like sentience, consciousness, and awareness, regardless of whether we ‘believe’ machines will or will not have them. It is foolish to assume that they absolutely never will or can, especially given what the technology can do today, in its very nascent stage (we must bear in mind that just two years ago from this point of writing, ChatGPT and what it can do today, was not something we would really have believed either. Technology moves quickly and as Amara’s law states, we humans are prone to underestimate its long term effects!)
The paper is not an easy read, but consciousness is not an easy topic. The easy way out is not always the best. We can all speculate on consciousness by watching or re-watching Star Trek episodes and writing long, philosophical, seemingly-deep paragraphs six ways to Sunday, but that’s creative fiction, not science. We need proper theory and formalism that we can debate and critique, and these authors are offering it.
I would be remiss in not pointing out that it is the authors’ view, not mine, that AI consciousness if ‘inevitable’. I don’t actually have a view on this, because I am divided on what it means to be conscious. I know I am conscious, and I believe most people I talk to are conscious, but I’ve never stopped to really think about what that means. I do have some training in philosophy, so I know that superficial understanding on a topic like this is not adequate for drawing rigorous conclusions. There are several proverbial cans of worms that can be opened when this topic comes under review. What this paper does for me is to get me to think about this issue more deeply. I still don’t fully understand it, and I suspect, not even the authors really do or have the final say on this matter, but they are closer to the heart of the matter than I am and I respect that.
So is AI consciousness inevitable? I would caution against taking the authors’ word for it, but I think that they are provoking us (intellectually) into answering why not. I hope someone smart and accomplished takes up that challenge and offers another side to the debate, perhaps another senior computer scientist with the same depth of knowledge as the authors!