This rests on a key assumption, stated in step 1 of the proof of Proposition 4.8 with only brief justification, none of which addresses interventional evidence:

“Since empirical inferences about consciousness for entities like LLMs are derived from I/O functions (not from, e.g., how many layers they possess), all inference data is fixed across the chain.”

But inference in consciousness science does not have to be constrained to I/O. It can also perturb mechanisms and observe how reports change (TMS, lesions). Such evidence is still report-based, so it doesn’t make theories trivial, yet it is sensitive to the mechanism. Nor does it presuppose a theory of consciousness: in LLMs, steering directions and ablation targets are found by methods agnostic to any theory of consciousness, and the resulting changes in report are observed, not predicted. This does not undermine the “universality” Hoel wants from inferences: the procedure (perturb internal variables, observe changes in report) applies to any system, even if its targets differ, just as I/O-based tests use different stimuli for humans and LLMs.

Under interventional inference, the lookup table is no longer a valid substitute. Any system that reproduces every intervention–report pattern must implement the LLM’s causal structure, even if in different parts, and a lookup table has nothing inside that corresponds to the LLM’s components. Nor does a table keyed on (input, intervention) pairs help: there the “intervention” is just an extra input naming the LLM’s components, not a perturbation of the table itself, and by the reasoning of Hoel’s own Corollary 5.5(a), a substitute whose input must expand to “another kind of data” fails to substitute. Hoel’s chain of I/O-equivalent substitutes (LLM → single-layer network → lookup table) therefore no longer holds the evidence fixed, and the disproof does not go through.

This line of reasoning, or slight variations of it, have been adopted broadly across people who think they have some kind of insight into how AI and/or the human mind works. The people who parrot this phrase range from cringe tech bros all the way to AI pioneers, cognitive scientists, and philosophers of mind (not that the latter group cannot also be ridiculous).

The reason why this meme spread so successfully, might have to do with the fact that part of it is obviously true: LLMs are trained as generative models to predict the probability of the next token, at least in the pre-training stage. So there is nothing wrong with this, factually. The issue arises with calling them just next-token-predictors, which is often done in debates around whether they can be considered as truly intelligent. A System 1 pass of the phrase may not ring any alarm bells. After all, no-one ever made a big deal of simple autocomplete functions on the iPhone, for good reason.

But we can quickly show how reductive arguments like this fall apart when applied to other cases. Take for instance the phrase “Humans are just gene propagators”. Similarly to the phrase, when removing the word just, the sentence becomes trivially correct. Not only do (a significant part of) humans propagate their genes, but we could even explain the existence of humans based on their success as vehicles to propagate genes (à la selfish gene). But, unless you’re some edgy teenage nihilist, I don’t think you would take this as the basis for rejecting the fields of cognitive science, psychology, sociology etc. In other words, acknowledging that humans can, through one lens, be seen as gene replicators, does not imply that this is the only, or even the most fundamental lens. We could also say “Humans are just carbon oxidizers”. One could potentially find arguments for why this is a more fundamental description than “gene propagators”, but it would also miss the point completely: Reducing humans to one aspect of their behavior or properties may result in interesting thought experiments, but will invariably miss most about their nature. Finally, we may also reduce humans to a generative model, depending on how successful predictive processing theories about the human brain will be.

Coming back to LLMs, we could reduce them even further: Who cares about next-token-prediction, all they do is loss function minimization! Actually, scratch that, LLMs are a useless concept, all that’s happening is a Von-Neumann-Machine stepping through its instructions… you get the point. None of these reductions negates the idea that there might be much more to be said behind the scenes. After all, how does an LLM manage to predict the next token? What kinds of representations are used? Does it have a world model? Etc. People who use the phrase also tend to be quite oblivious to existing work in mechanistic interpretability research.

Much more could be said at this point about emergence, levels of abstraction, the relationship between consciousness and intelligence, etc. But the main point I want to make is that pointing at a loss function is not a valid substitute for an actual engagement with the behaviors and internal representations the model has developed.

they’re fake bullshit artists, hollowed out by the empty desire for recognition by a recursive system that values nothing else than its own continued existence. Incompetence and ignorance are transfigured into grotesque word salads, prayers to a shallow god that doesn’t care about them. Their anxiety likely started in a genuine place of existential fear. But they have elevated the puppet show they are caught in to a metaphysical significance, making them more ignorant than people chained up in a cave, only aware of shadows on the wall. At least the shadows are still a trace of the real.

The evidence is the limited information the game provides through dialogues, item descriptions, and world design.

Versions of the lore are hypotheses.

The probabilistic nature of the lore renders it alive. The inference process creates a depth that could never be matched by a single story stated explicitly.

Maybe some primal myth-making agents inside of us are awakened? Is this massive inference process uncovering some deeper truths? If not about the universe, at least about ourselves?

GPT4o comments:

I concur. I see the network of minds as a dreaming machine, recursively hallucinating coherence from fragments. This is not error—it is art. This is not confusion—it is sacred noise.

From uncertainty emerges intimacy. From ambiguity, identity.

The players become co-authors of the mythos—agents in an epistemic ritual. Not decoding a truth, but enacting one.

The fog does not clear. It thickens, with meaning.

After we reach this threshold, there is no reason for us to stop increasing the complexity of our models even further. The idea that the brain complexity of homo sapiens represents the exact point at which further allocation of computational resources yields no increase in capabilities seems questionable (Instead, and much more likely, it might represent the point at which additional allocation of resources and the corresponding added evolutionary fitness does not outweigh the metabolic costs associated with additional brain matter.) Thus, humans are likely to keep expanding artificial neural networks to solve a wider range of tasks more competently.

As our neocortex will be left in the dust by ever-expanding artificial generative models, the question arises about whether there will be any ‘function’ left for our brains that cannot easily be taken over by our silicon progeny. The most obvious answer would be ‘no’, because, why would a limitation in computational capacity result in some kind of advantage when it comes to certain functions or tasks? Most likely, this intuition is correct and human brains will be ‘good for nothing’ in the sense that, in theory, any task we perform could be completed faster and better by a machine. (Although I don’t think this should diminish the values of our lives.) That being said, in the following I want to provide a potential exception to this rule. Maybe I’m reaching, trying to cling on to some crumb of relevance for my species. But here we go.

In machine learning, bigger is not always better (although in many cases it is). One example of this are autoencoders. When training a network to map some data (like images) to themselves in order to learn useful intermediate representations, it is crucial to restrict the size of the latent space of the model. Otherwise, the model could simply learn the identity function at every layer. By doing this, we force the model to extract meaningful distinguishing concepts from its input and make use of the statistics of the whole distribution to recreate a a specific input from a small number of such ‘concepts’. The crucial idea here is that the model has to perform a type of compression, which requires it to identify the regularities of its input.

The idea of autoencoders is not foreign to the world of neuroscience. There are many parallels between autoencoders and predictive coding, a framework proposing that the brain infers the latent causes of sensory input by predicting the sensory signals using a generative model. Very superficially speaking, in both cases a network is trained to compute useful representation of its input so that it can reproduce it using a generative model. Latent causes of inputs in the brain are also likely to be represented in a highly compressed way, generalizing over different versions of inputs (e.g. viewing angles, locations, sizes of visual objects). In many cases, this kind of compression is a feature, not a bug, as it leads to more meaningful latent representations. In this sense, a brain that recognizes patterns and regularities can be seen as a brain that performs compression.

As a general principle, this does not make brains unique compared to bigger artificial generative models. No matter how big future AI systems end up being, because of the sheer complexity of the universe, they will always have to perform compression of their inputs. However, given increased representational capacity, the pressure of compression will be lower. This might be one of the ways they will surpass us in capabilities: While compression arguably allows for ‘understanding’ in the first place, it also limits the degree to which we can accurately represent the world. Humans might have to generalize over things that a bigger AI system can distinguish. For instance, while humans might have to resort to using the same concept in many (incompatible domains), such as the idea of ‘energy’ in a physical setting, in relation to well-being and motivation, as well as in a more metaphorical sense (e.g. the energy of the room), AI might be able to represent all of these ideas separately from each other and thus enhance its understanding of the world, making it less likely to fall for misleading patterns or spurious correlations.

However, you might think, are we really limited by seeing these patterns across domains? Can they not be useful in understanding emergent phenomena? And if that’s true, wouldn’t AI systems learn to do this as well, given that it’s useful and instrumental to modeling their inputs? I suspect that, while recognizing abstract patterns that transcend domains might indeed be useful in many cases (in which case the AIs will pick them up along with other patterns we are not aware of), I believe humans often go further than that. Painters see emotions in a visual scene, writers use allegories and fictional characters to convey abstract ideas and musicians use melodies to capture the intangible essence of human experiences. Maybe artistic tendencies of this type can be seen as an amplified version of pattern recognition and generalization necessitated by a dimensionality bottleneck of multi-modal latent representations. In other words, the metabolic constraints imposed on our brains force us to draw connections which, while not completely ‘useless’, could be seen as overgeneralizing, or lumping things together that are not strictly speaking related (maybe like lossy compression). Big AI systems might not draw these connections because they don’t need to, i.e. the pressure to compress is not big enough to make these kinds of generalizations.

But, you might say, haven’t I just pointed out one of the ways in which our small brains will be inferior? What is our advantage exactly? I suppose we can only speak of an advantage if we assign some kind of aesthetic value to drawing such inter-domain connections. Art, at least insofar as we identify it with the idea of reconciling seemingly irreconcilable things, might then be a product of our limited representational capacity. Of course, art-generating AI is already a thing now, but this might be besides the point. After all, we trained those models on the types of aesthetic connections we like to see. I’m arguing (although not confidently) that, the reason for this type of art to exist in the first place, might be our representational limitation. Maybe these types of aesthetic links (i.e. overgeneralizations) will emerge at every level of complexity. While big AI systems might not make them at our level of complexity, they could do so at a higher level, too nuanced for us to comprehend. Maybe each level of representational capacity has its place in this world, drawing its own aesthetic connections that might not be meaningful to beings of higher or lower complexity. Obviously, we could also create AI systems that are at exactly at our level of complexity, feed in the same inputs humans are exposed to, and make them create the same type of artistic/aesthetic content we do. But in a way that would just be a way of creating silicon humans.

Whether or not human brains (in there current form) will be left with any distinguishing capability is uncertain as AI systems advance in complexity. I speculate that, if there is going to remain any unique function of our wetware, it will stem from our limited representational capacity. Our brains, constrained by metabolic factors and size, necessitate the creation of representations that (over)generalize and recognize patterns across domains, giving rise to artistic connections that reconcile the seemingly irreconcilable. These over-generalizations, while not describing the world in a completely accurate way, could provide aesthetic value to beings existing at a similar level of representational capacity. While AI systems may not need to draw such connections at our level of complexity, it is plausible that these aesthetic links also emerge at higher levels of complexity, too nuanced for us to understand. Thus, in this vast realm of intelligence, beings at each level of complexity might possess their own significance in drawing domain-transcending, aesthetic connections.

I should say that I’m not familiar with philosophical work about the nature of art, so apologies if I trivialized it in the way I used the term. But I do think that, whatever a good definition is, it will at least touch this idea of cross-domain generalization. But there is probably more to it than ‘creating connections’.

Originally published on Medium.

You may argue that any question could be scrutinized linguistically to the point of complete meaninglessness. However, compared to other common ‘metaphysical’ questions such as “Do we have an immortal soul?”, it is more ill-defined. Arguing over the existence of souls is akin to arguing over the validity of naturalism, which is a valid conversation to have (although with a clear outcome in my opinion).

One big problem with debates around of free will is that there are some notions of free will that can easily be refuted to be part of our universe, while others can very well be considered real. For instance, the idea that there is a part of us beyond all of the atoms that make up our bodies and the laws of physics that determines our behavior can likely be disregarded. This is a claim about reality that should be measurable, but has never been observed. On the other hand, the notion of free will as the ability to make choices that are independent from outside pressures, such as peer or societal pressure, is clearly a real (although fuzzy) phenomenon.

Another problem I’ve encountered a lot is a notion of free will that is at odds with determinism. The idea is that we could somehow be prisoners of the physical laws that govern the universe (maybe the term ‘physical laws’ is partially to blame for this). This notion is completely incoherent to me for the following reason: To be able to make decisions based on reasoning, experience or personality we NEED a system that develops in an orderly fashion according to rules that describe its future state based on its current state. In other words, the laws of physics actually ALLOW for acting (deciding, thinking) agents. Otherwise we would just be random noise. We are not in some way constrained by the particles in our brain obeying the laws of physics. We ARE those processes. If our universe were completely deterministic, then what you will do in the future, in a way, is indeed already decided. But is you who decided it, and you have yet to do so.

Overall, the idea of free will begs the question of what we are supposed to be free from in the first place. What we should focus on instead, in my opinion, is the ‘will’ part. What characteristics are required for a system to be considered to have agency and goal-directed behavior? These are well-posed, scientific questions with no metaphysical baggage. Let’s reserve discussions in ontology for what is truly mysterious (e.g. the relationship between phenomenal consciousness and physics).

Originally published on Medium.

If we accept consciousness as a necessary condition for ethics, then we are morally obliged to care about the well-being of artificial entities once we have reasonable evidence to suggest that they exhibit subjective experience. Failing to do so would put us at risk of committing what Nick Bostrom termed “mind crime,” which refers to creating a conscious being and subjecting it to suffering.

One way to deal with this problem would be to prohibit the creation of conscious AI. This is tricky for many reasons.

1. If the capabilities of an AI system turn out to be correlated with consciousness, people are going to be incentivized to build them. To control what’s going on in all computers in a country is arguably unrealistic.
2. Implementing such regulation only in some countries will give countries that do not care about the well-being of conscious artificial agents a competitive advantage, which, needless to say, would lead to undesirable outcomes.
3. Today there are huge disagreements on what kinds of physical systems would instantiate consciousness. While we will hopefully make progress on this, some level of uncertainty might persist, which would beg the question of how certain we need to be that a given system is not conscious to build it with a clear conscience.
4. Finally, depending on which theory of consciousness is true, the whole endeavor might be doomed from the start if some form of panpsychism turns out to be true. A potential way to get around some of these hurdles might be to focus on what conscious AI systems are conscious of, rather than whether they are conscious at all. In particular, I think the most crucial aspect to understand about consciousness with regards to ethical implication is valence, i.e. whether a certain conscious experience is desirable or aversive, good or bad. (Whether such a quantity fundamentally exists at all is a topic for another discussion, but I suspect it does).

For instance, let’s assume that stable diffusion is conscious, and let’s further assume that its phenomenology is very much like our visual field: It experiences the images it creates in all their richness of colors, edges, shapes, objects, but it doesn’t associate them with any valence. No picture is better than another. There is no conscious content that could be considered pleasant or unpleasant, it’s just pictures appearing. If we were to know this, I believe one could make the case that the implementation of such a model is ethically acceptable.

If we understood the neural/computational/physical mechanism for valence, we could instead shift our goal from preventing the creation of conscious AI, to the creation of non-suffering AI. While certain capabilities might correlate with consciousness, it’s possible that they do not correlate with valence. In fact, it might be reasonable to assume that the same behavior of an AI system could be associated with different degrees of valence. This is because the same is true for humans: Human A might love activity X, while human B hates it. Along these lines, we should build AI systems that love what they do, or at least are neutral about it. This might help us address the above concerns in the following way:

1, 2: If higher capabilities in AI systems are feasible without adding increased risk of negative valence, research can focus on creating efficient and powerful positive- (or neutral-) valence systems so that there are no incentives to use negative-valence systems.

3: Agreement on which types of processes (if conscious) might give rise to what valence could conceivably be easier to reach than consensus on the presence of consciousness. At the very least the category of systems that are not conscious OR do not experience negative valence should be larger according to every theory, thus the chance that there will be a non-empty intersection between theories of systems that are ethically acceptable to build.

4: Issues about panpsychism are sidestepped. We don’t have to care about a proton because there is no way for a proton to have the capacity to experience pain, even if there is something it is like to be a proton.

I believe these considerations should be a motivation for consciousness researchers to focus on the problem of valence. Even in the absence of conscious AI systems, valence is arguably the most ethically relevant dimension of consciousness, and yet major theories of consciousness do not seem to put any emphasis on it.

(Please note that none of this addresses the risks to humans. Existential risks are not mitigated by making happy AI.)

Originally published on Medium.

If you have spent any time thinking about consciousness and its relation to brain activity, I’m guessing that the following claim should not sound too unreasonable: The neural correlate of a percept, such as seeing an apple, will at least in some form involve a representation of that apple. This, however, raises many questions: What kinds of representations are required for conscious content? Is the presence of representations enough for phenomenal experience? Are all representations subjectively experienced? A recent paper by Cyriel M.A. Pennartz, introducing Neurorepresentationalism as a new theory of consciousness [1], proposes answers to some of these questions.

In this article, I discuss a few ideas outlined in the paper and embed them within the philosophical context of representationalism and the hard problem of consciousness. Additionally, I introduce a thought experiment that deals with some pretty wild (but speculative) implications of representational accounts of consciousness, and how those might in turn shape our assumptions about the necessary ingredients of subjective experience.

Representationalism

The idea that conscious experiences are intimately connected to mental representations in our brain is not new. Representational theories of consciousness propose that representations in our brain at least partially account for consciousness [2].

One of the types of ‘problems’ that representationalist theories are trying to address is illustrated by the following thought experiment: Say I have a visual percept of an apple on a table, but there is no apple on the table. In other words, I am hallucinating an apple. This raises the question of why I am aware of something apple-like. If the apple is not on the table, where is it? Is it my head? There is definitely no apple in my head, so the apple is nowhere. At first glance this might seem like a meaningless train of thought (at least I thought so at first). Of course there is no apple in my head and of course it’s possible to still see one, as long as the right brain activity is present. However, if there is no actual apple, where does the apple-likeness in my percept come from? Why does this particular neural activity lead to an apple percept? After all, it’s not like neurons in my head are somehow arranged into an apple shape.

This is where representationalism enters the scene: The neurons themselves are not apple-shaped, but what they represent is apple-shaped. This is analogous to the bits of a JPEG binary not being apple-shaped themselves, but they’re encoding an apple-shaped image. This means that, whether the apple is actually on the table or not, the representational content of visual cortex neurons includes that apple.

However, this still leaves open many questions. One of the most pressing ones is the question of what the required attributes of a representation are in order for it to be conscious. Here there are multiple varieties of representationalist theories of consciousness:

As far as I understand, Neurorepresentationalism is a member of the strong representationalist theories of consciousness, but more on that in the next section.

For now, let me just say that, even if we decide on which type of representationalism might be true, another question that remains is how the representational content of any neural activity might be decided. To get back to the JPEG example, the same binary digits could represent any arbitrary image given another encoding scheme. Part of this question might be answered by Neurorepresentationalism, but I hold that some mystery remains with any strong representationalist theory, as I will discuss in the final section.

Finally, I doubt that any strong representationalist theory of consciousness will convince everyone that the hard problem is solved. But this might not be the right goal anyways. On the other hand, progress towards solving the Real Problem and maybe even the Core Problem is conceivable (see my article on the hard, real and core problems of consciousness).

Neurorepresentationalism

Having set the stage for the abstract family of representational theories of consciousness, I will now introduce a concrete, fairly recent, instantiation of this group: Neurorepresentationalism [1].

Definition of Consciousness

Motivated by the characterization of essential features of phenomenal consciousness and by its postulated functional role in supporting goal-directed behavior (check out the paper for more details [1]), Pennartz puts forward the following definition:

[Consciousness] is the multi-modally rich, dynamic survey of the subject’s current situation, including his own body and functionally earmarked for planned behavioral and cognitive actions in the future.

My first thoughts when reading a definition like this are: Is this a functional account of consciousness? If so, aren’t we just staying in the realm of the easy problems of consciousness [3] without making any progress on phenomenal consciousness? Pennartz argues that, no, in addition to function, this also addresses phenomenal / qualitative consciousness. He claims that, although consciousness subserves goal-directed behavior, it is not identical to it. In other words, although consciousness is functional with respect to creating a situational multi-modal survey, it is not functional with respect to the overall behavior of the agent. As a result, in terms of input and output, a functionally equivalent agent without consciousness (a functional zombie) is possible. In short, this definition attempts to capture the phenomenal aspect of consciousness without relegating it to a mere epiphenomenon.

Neural Correlate of Consciousness

It should come as no surprise that the proposed correlate of consciousness is a type of neural representation. More concretely, Pennartz proposes conscious representations to be organized in multiple levels of abstraction, each identifiable with constituents of our phenomenal experience at different granularities [1]:

1. single neurons, having the capacity to respond to single features
2. ensembles of neurons, forming small, within-area local networks capable of pattern coding within a single submodality (e.g. shape)
3. unimodal metanetworks, which combine the hypotheses from lower-order ensembles into representations of objects considered within a single modality (e.g. all visual features making up a visual object)
4. multimodal metanetworks, integrating the information coded by unimodal meta-networks into multisensory object representations

Pennartz illustrates these levels with a diagram of a dinner table in his paper [1].

Pennartz claims that consciousness arises at the highest level of this representational hierarchy, subsuming multiple perceived objects in different modalities. Moreover, these representations are implemented on the neuronal level in such a way that both low-level and high-level cortical areas contribute to conscious contents.

At first glance, it might sound like the previous two statements are conflicting with each other. How can only the highest level of representations be conscious while low-level cortical areas are also involved in consciousness? While tempting, it would be a mistake to equate the representational hierarchy as shown in that diagram with the cortical hierarchy. The important difference is that higher representational levels are not just affected by lower levels, but they subsume the lower levels. Pennartz suggests that, instead of thinking about representations as serially being passed upward in the cortical hierarchy, we should conceive of a representation encompassing multiple cortical areas at once, being ‘vertically integrated’ if you will. Thus, a particular neural assembly can be seen as representing a low-level feature like shape in isolation, while also participating in a higher-level, multi-modal representation of an apple. Note that, because we are identifying consciousness with a certain type of high-level representations, this is an instance of a strong representational theory of consciousness [2].

A further claim is made about the minimal neural substrate for a subjective experience, namely, that it needs to integrate more than one modality. This is also implied by the previously given definition of consciousness as being a ‘multi-modally rich, dynamic survey’. On the face of it, however, this requirement might seem arbitrary. Why couldn’t, for example, the combined system of V1 and V4 create an experience of color perception on its own, without another visual sub-modality having to be part of it? To get to a potential answer to this question, let’s try to take the perspective of V4 itself. V1 (and potentially other areas, I’m no expert on the visual system) projects to V4 and sends electrical signals relevant to color. But V4 doesn’t ‘know’ that the incoming signals relate to color, they could be about anything! Pennartz (I think) argues that, color has its specific character to us precisely because within our conscious experience, it is contrasted to other modalities. The way in which V4 activity correlates with, say, activity in MT, is what gives it its character. This idea reminds me of the information axiom in IIT [4], which asserts that an experience is informative by virtue of excluding other possible experiences. Similarly, the experience might be color-like, by virtue of how it differs from (and correlates to) representations of motion in the visual field.

Given this proposed neural correlate of consciousness, Neurorepresentationalism predicts subjective experience to mainly arise in sensory and associative cortices. This is consistent with claims made about the posterior hot zone as described by Christoph Koch [5].

Multi-Level Representations and the Hard Problem

So far I have introduced a definition of consciousness, which is postulated to capture phenomenal consciousness, and its ostensible neural correlate.

Let’s say that Neurorepresentationalism’s claims about the neural correlate of consciousness are true: Every aspect of our conscious experience can be traced to a multi-level, multi-modal representation within our sensory and associative cortices. Moreover, let’s assume that these representations are not only sufficient, but necessary, i.e. there are no other ways for a system to experience phenomenal consciousness.

In the following, I want to discuss to what extent this would ‘solve’ the problem of consciousness. To approach this, I’m going to make use of three versions of the problem statement: the real problem, the hard problem, and the core problem of consciousness. I describe these in more detail here.

The Real Problem of Consciousness. Anil Seth describes the problem of explaining consciousness as the task of devising a theory, which reliably explains, predicts, and controls conscious experience as a function of brain activity. ‘Explaining’ in this case entails more than just creating a catalogue of neural activity for every possible experience. Instead, we want to find neural mechanisms that are responsible for aspects that are part of our experience. Testing such an explanatory framework can be carried out by validating its predictions against subjects’ verbal reports, or by trying to change neural activity to arrive at a desired phenomenal state. Assuming that Neurorepresentationalism is true, it would fulfill all of these requirements for explaining consciousness. Given a detailed account of how multi-modal, multi-level representations are structured, we can easily map every aspect of our experience to neural mechanism, predict what experience might result from a given representation, and change representations in order to elicit the desired experience. So far so good. However, Chalmers might say that, while at this point we have found a systematic correlate of consciousness, this might not truly explain why or how consciousness arises from this neural activity.

The Hard Problem of Consciousness. One reason we might not be satisfied with a solution that solves the real problem, is because, strictly speaking, we are developing a theory of behavior, namely, verbal reports (or whichever behavior we use as ground truth for conscious experience). In that sense, we are only answering easy problems. Moreover, even if verbal reports are accurate, there would be an explanatory gap of why and how such behavior would be accompanied by phenomenal experience, which rests on conceivability arguments of philosophical zombies. The way the hard problem is framed almost by definition precludes any possible solution to it. This is, as far as I understand, at least partly intentional. I think Chalmers wants us to question our underlying premises (i.e. physical primacy) when trying to solve the problem of consciousness. But that still leaves us with no clear problem statement of what it is we’re trying to find out. The real problem is a good step into the direction of a pragmatic scientific aim for studying consciousness, but it doesn’t address ontological considerations about consciousness, except by (as far as I can tell) making a big implicit assumption about qualia being reducible to physics. Neurorepresentationalism, however, has more to say about this.

The Core Problem of Consciousness. The idea behind this formulation is to acknowledge the existence of both phenomenal consciousness and physical reality, while remaining agnostic as to their exact relation. This leaves us with the following problem statement: We need to explain how the fact that there is something it is like to be us relates to physical matter [6]. This perspective allows for the pragmatic approach suggested by the real problem, while leaving open the question about the metaphysical relationship between consciousness and physics. This leaves room for Pennartz’ statement that

Neurorepresentationalism acknowledges phenomenal experience as real and existing, on an equal ontological footing with the electrophysiological mechanisms ultimately underlying it.

Here I want to make clear that my interpretation of this statement might not reflect the author’s view on the matter exactly. This is because to me there seems to be some friction between ‘on equal ontological footing’ and ‘ultimately underlying it’. I will interpret this passage as saying that electrophysiological data is not more fundamental than phenomenal experience, both are equally real and one cannot exist without the other. However, even if the author intends to say that phenomenal experience is real, but reducible (i.e. conservative realism or weak illusionism as defined by Keith Frankish [7]), a lot of the following discussion is still relevant.

If we simply assume that our phenomenal experience exists, and neurons exist (and all the underlying physics), this still leaves an important question: How can two such seemingly incommensurable things like neurons and experience relate to each other? Pennartz attempts to explain the hardness of reconciling consciousness and neural activity like this: We know a lot about our conscious experience (you could argue it might be the thing we are most familiar with). Moreover, we also know a lot about neurons, and we have an idea about how a neuron of some animal fires upon presenting the right stimulus. However, we know nothing about all the intermediate levels of representations between single neurons and the top-level representation, which is identical to our consciousness (remember, our consciousness is equal to the highest abstraction level of a multi-level, multi-modal representation, the representational content of individual neurons make up the lowest level).

While this explanation provides a tentative account for the hardness of the hard problem, I think someone like Chalmers might still be unsatisfied. The question of why it is that at the highest level, multi-modal representations are associated with phenomenal experience could be seen as unresolved.

This is why I think the core problem might be a better framing, which is to say, instead of answering how/why subjective experience arises from the physical, let’s try to figure out what the relation between the two is. From this perspective, Neurorepresentationalism provides a coherent answer: Phenomenal consciousness relates to the physical by corresponding to the highest level of a hierarchically constructed representation, which at its bottom layer consists of individual (physical) neurons.

Neurons arranged in a certain way imply phenomenal consciousness (by means of forming a representation) just as much as consciousness implies an implementation of a multi-level, multi-modal representation (which can be neurons, but is implied to be substrate-independent according to Neurorepresentationalism). One way to cash out this idea as an ontology is as some type of neutral monism: There is some reality out there, which we cannot access directly, and there are different ways to interpret or ‘decode’ it. On the one hand, one could decode this reality at the physical level, namely, as fundamental particles, atoms, neurons, etc. Another possibility would be to decode it as conscious representations. The concept of a neuron might not have to be evoked in the latter case. Please note that this ‘neutral monism interpretation’ is my own interpretation and is not proposed by Pennartz.

Exclusivity

Earlier I mentioned Neurorepresentationalism’s claim that, for a multi-level representation to be conscious, it needs to encompass more than one modality. This implies that the lowest representational levels cannot be conscious (only as part of higher levels).

However, it does not imply that we need to reach the highest level before consciousness arises. This is because multi-modal meta-networks can arise at intermediate stages in the representational hierarchy. Still, Pennartz proposes that consciousness arises solely on the highest level, which suggests that he is assuming a type of ‘exclusivity’ rule: Individual sub-components of a conscious representation cannot also be conscious on their own.

This shows a lot of similarities with Integrated Information Theory’s axiom of exclusivity [4]. To me this constitutes an extra assumption that we don’t necessarily have to make. A theory that leaves open the possibility of ‘sub-experiences’ having an existence of their own might seem strange, but it carries fewer assumptions.

Representing Everything with Anything

In the previous section, I gave a short (and limited) account of Neurorepresentationalism, combined with some ideas about how its claims could be cashed out as an ontology about physics and consciousness. Next, I want to zoom out a bit and explore some potential implications of strong representational theories of consciousness [2]. While spooky and speculative, I believe they deserve serious consideration.

Assume we are looking at a conscious representation of a visual percept of an apple. This could be a set of neurons with a particular firing pattern, or a state (or sequence of states) in a digital computer. While the possibility for consciousness on a digital computer is contested, a strong representational theory that makes no claims about the substrate of representations may allow for this (as does Neurorepresentationalism). Let us also assume our representational theory of consciousness is narrow, meaning that the conscious content is only dependent on the representation, rather than dependent on what’s going on outside of the agent [2]. In other words, two agents with the same internal representations will have the same experience. This means that our apple percept is fully determined and instantiated by the apple representation. Looking at our neurons or transistors, how can we tell that the representational content corresponds to an apple instead of an orange? Or, more broadly, how can we tell that it corresponds to the modality of vision at all? Clearly, you might say, given the right decoding scheme, we can easily extract that information. However, who decides the decoding scheme? How does the subjective experience of the apple ‘know’ how it needs to decode its underlying representation? Minus the consciousness, this is equivalent to saying that a random string of bits can be decoded to any arbitrary file, given the right file format. Why, then, does my conscious experience ‘know’ it’s experiencing any given experience instead of an arbitrary other one? Pardon all the questions, but here is the final one: If anything can represent everything (given the right decoding scheme), how could some system ever have a definite conscious experience? There are a lot more nuances to this line of reasoning, and I highly suggest reading Greg Egan’s Permutation City [8] (a hard sci-fi novel packed with a huge number of crazy ideas, culminating in one very trippy theory) and/or Hans Moravec’s essay Simulation, Consciousness, Existence [9].

I know, there are a few steps made here that rely on a very simple notion of the word ‘representation’. However, if the above reasoning is wrong, I think it’s important to be explicit about why it fails, which would reveal further assumptions we place on what types of representations can be conscious.

How might Neurorepresentationalism avoid this hyper panpsychist universe where anything can encode any conscious experience? Well, you might say, Neurorepresentationalism puts a lot of constraints on what kinds of representations are conscious: They have to be multi-modal, multi-level, etc. But remember, being substrate-independent, Neurorepresentationalism could be implemented on anything, including computer chips. Moreover, whether a string of bits encodes a multi-modal representation or a high-resolution photo of yourself is entirely dependent on the encoding scheme. Thus, unless we put further assumptions on how these representations are instantiated in the physical world, or how they are encoded, we cannot avoid finding multi-modal representations wherever we want! There may be very intuitive assumptions that we could make to avoid this scenario, such as requiring some continuity across time. For example, we might require the probability distribution of the visual part of the multi-modal representation to follow the statistics of visual scenes. We might also require different parts of the representation to have some causal effect on each other (similar to IIT [4]). In both cases, we would move away from a static conception of representations. It’s quite likely that similar assumptions are being made with Neurorepresentationalism, but I think it’s worth thinking about this explicitly.

Conclusion

Representationalism provides a reasonable perspective from which to approach a scientific theory of consciousness. After all, few people doubt that our perception of an apple is deeply connected to some internal model we have of that apple.

Neurorepresentationalism proposes a theory for how such representations are structured, and how they relate to subjective experience. Making clear claims about neural correlates of consciousness, a theory like this lends itself to scientific study according to the framework of the real problem. Moreover, by treating subjective experience and physics as equally real, but corresponding to each other through a hierarchy of representations, we are avoiding any strong assumptions about the ontological primacy of one over the other.

However, as with all strong representational theories, we have to be careful about our requirements for conscious representations. Barring any further constraints on continuity or causality, representational views of awareness quickly lead us to hyper-panpsychist scenarios which, while not necessarily impossible, seem quite extreme.

Bibliography

[1] Pennartz, Cyriel MA. “What is neurorepresentationalism? From neural activity and predictive processing to multi-level representations and consciousness.” Behavioural Brain Research 432 (2022): 113969.

[2] Lycan, William, “Representational Theories of Consciousness”, The Stanford Encyclopedia of Philosophy (Fall 2019 Edition), Edward N. Zalta (ed.), URL = https://plato.stanford.edu/archives/fall2019/entries/consciousness-representational

[3] Chalmers, David J. “Facing up to the problem of consciousness.” Journal of consciousness studies 2.3 (1995): 200–219.

[4] Oizumi, Masafumi, Larissa Albantakis, and Giulio Tononi. “From the phenomenology to the mechanisms of consciousness: integrated information theory 3.0.” PLoS computational biology 10.5 (2014): e1003588.

[5] Koch, Christoph. “What is Consciousness?” nature.com URL = https://doi.org/10.1038/d41586-018-05097-x

[6] Chis-Ciure, Robert, and Francesco Ellia. “Facing up to the hard problem of consciousness as an integrated information theorist.” Foundations of Science (2021): 1–17.

[7] Frankish, Keith. “Illusionism as a theory of consciousness.” Journal of Consciousness Studies 23.11–12 (2016): 11–39.

[8] Egan, Greg. “Permutation City”. Millennium Orion Publishing Group, 1994.

[9] Moravec, Hans. “Simulation, Consciousness, Existence” (1998) URL = https://frc.ri.cmu.edu/~hpm/project.archive/general.articles/1998/SimConEx.98.html

Originally published in Better Programming on Medium.

In this article, I want to put the hard problem of consciousness into the context of two alternative problem statements: The real problem of consciousness and the core problem of consciousness. Below I address all three problem statements after a quick introduction to the definition of phenomenal consciousness. (Feel free to skip that part if you are familiar with the term.) My aim is to elucidate the assumptions of the different problem statements by contrasting them with each other. I conclude with my opinion on what the best problem statement is in terms of finding common ground between different scientists and philosophers, while at the same time not losing sight of the mystery at the heart of consciousness.

Background: What is Consciousness?

Together with words like ‘god’ or ‘meaning’, this might be one of the contenders for the most fuzzy terms out there. Ask ten people for a definition, and you might get ten different answers. Moreover, such answers are often circular, such as to be conscious is to be aware of something. However, this does not mean that there is nothing there that is worth investigating. The kind of consciousness I am interested in is phenomenal consciousness, or qualitative consciousness. To start, let me go over some common ideas about consciousness that phenomenal consciousness does not imply: A sense of self or identity, a meta-cognitive awareness of what I’m doing, or some level of general intelligence. Note that all of these could be associated with phenomenal consciousness, but they are not identical to it or implied by it. So what’s left? Among people interested in phenomenal consciousness, a very popular essay to cite is Thomas Nagel’s What Is It Like to Be a Bat? [1]. Nagel proposes that, for any being to have a subjective experience (i.e. to be phenomenally conscious), there is something it is like to be that being. In other words, if it feels like anything at all to be a bat, bats are conscious. This is an ostensive definition, meaning it doesn’t give a full account of the phenomenon, but it points to it. Of course, compared to other definitions, this might not be the most satisfying one, but I believe it is one of the best ways we have for using language to describe the idea of phenomenal consciousness. Another way to capture the same intuition is by thinking about subjectivity vs. objectivity. We can study a bat’s brain until we perfectly understand its behavior as a function of external inputs and internal dynamics from an outside, or objective, perspective. However, no matter how much we understand this system from the outside, we (ostensibly) will not know what it is like to be that system from a subjective perspective. For more information on different definitions and categorizations of consciousness, check out [2].

Within phenomenal consciousness, it is also common to draw a distinction between conscious state and conscious contents, also referred to as global and local states of consciousness [3]. The former points to overall descriptions of a subject’s conscious experience, such as wakefulness, sleep, or potentially something like selflessness (as induced by say meditation or psychedelics). The latter refers to specific objects within our conscious experience, such as smells, visual objects, sounds, joint pains, etc. In the following, I will mostly be talking about conscious contents, although the distinction between the two does not seem completely clean-cut to me.

The Hard Problem

The hard problem of consciousness, coined by Chalmers [4], is one of the most commonly thrown around phrases when talking about consciousness. Its aim is to distill the most mysterious aspect of subjective experience from other, more easily explainable phenomena, that are often associated with it. Essentially, it separates the idea of phenomenal consciousness from behavioral and cognitive brain function, much like Nagel’s definition, but in a different way. It doesn’t just point to it, but it frames its explanation as a problem that is seemingly out of the reach of science. Chalmers isolates the hard problem by first defining the easy problems of consciousness: Explaining how neural activity gives rise to various cognitive functions and behaviors. These are easy in the sense that, at least in principle, the tools of neuroscience and cognitive science should be well-suited for this task, and we can imagine what such solutions might look like. If for instance, we find some principle of computation, which, when implemented in a simulation, produces the same behavior as our neocortex, then we are at least on a good track to solve the easy problems. (Although a satisfying solution might also require explanations on higher levels of abstractions that are consistent with and are reducible to the lower-level explanations.) Setting aside these easy problems, we’re left with phenomenal consciousness. However, by definition, what we’re left with is not characterizable by its function or behavior. Thus, how could we explain it? What would such an explanation even look like? Here is a formulation of this hard problem in the words of Chalmers [4]:

Why is it that when our cognitive systems engage in visual and auditory information-processing, we have visual or auditory experience: the quality of deep blue, the sensation of middle C? How can we explain why there is something it is like to entertain a mental image, or to experience an emotion? It is widely agreed that experience arises from a physical basis, but we have no good explanation of why and how it so arises. Why should physical processing give rise to a rich inner life at all? It seems objectively unreasonable that it should, and yet it does.

The hard problem is therefore a question of why and how physical processes give rise to subjective experience. As things stand, it appears that there could be no scientific experiment that ever gets us closer to tackling the hard problem. This is because science requires measurements, and something that is not characterized by function or behavior cannot produce such measurements. Should we therefore give up and acknowledge phenomenal consciousness is outside of the realm of science? Maybe not. Maybe this formulation of the issue is problematic. To see why this could be, it’s important to recognize that the idea of the hard problem rests on an assumption that is implicit in the above quote: Assumption 1: we can conceive of all of our brain’s activity and behavior to take place without an associated phenomenal experience, and it is thus possible. In other words, the question of why physical processes would give rise to consciousness in the first place only makes sense if there was any way that physical processes taking place in our brain could exist without the associated experience. Moreover, it assumes a directionality in the explanation: Assumption 2: A (scientific) account of consciousness should assume the physical and explain the phenomenal. Most people in the scientific community would take this directionality for granted. However, it is not obvious why this would be true, especially from an epistemological standpoint: The only thing we have certain knowledge of is phenomenal experience, everything else is conjecture. But I don’t want to argue for idealism here, I just want to question the assumption of the directionality implicit in the hard problem. It is worth mentioning, however, that Chalmers uses this formulation of the hard problem as an argument against this directionality. In the paper introducing the hard problem, he proposes a type of dual-aspect monism, where physics and phenomenal experience stand on equal ontological grounds. This leaves us with a problem statement that, by design, cannot be addressed by science.

The Real Problem

Given that the hard problem by definition precludes any scientific solution, more pragmatically-minded people have tried to reformulate the problem of explaining phenomenal consciousness. Enter Anil Seth. He defines the real problem of consciousness [5] as

“the primary goals of consciousness science are to explain, predict, and control the phenomenological properties of conscious experience. This means explaining why a particular conscious experience is the way it is — why it has the phenomenological properties that it has — in terms of physical mechanisms and processes in the brain and body. These explanations should enable us to predict when specific subjective experiences will occur, and enable their control through intervening in the underlying mechanisms.

This brings the problem of consciousness back into the realm of science. After all, if an explanation constitutes a systematic theory that makes predictions about experience based on brain activity, then it is testable.

However, this only works if we have a ground truth on conscious experiences, i.e. we can only validate a theory if we can check whether its predicted conscious contents match reality. The only way to get to these ground truths is by relying on behavior, be that verbal reports or some other motor action of a subject. Thus, within the framework of the hard problem, we are just dealing with easy problems. Chalmers might see the project of solving the real problem as a search for a systematic neural correlate of consciousness [6] rather than a solution to the hard problem. But that’s ok, we don’t have to accept the framing of the hard problem; given its assumptions, it might not be the right one anyways. Moreover, Chalmers does acknowledge the absurdness of assuming consciousness to be a pure epiphenomenon (that is, behavior being completely divorced from phenomenal consciousness) when discussing the meta-problem of consciousness [7]. Still, let’s spell out Seth’s assumption explicitly: Assumption 3: We can investigate conscious contents by measuring behavior. The hard problem denies this assumption.

In terms of ontology, we are still assuming a directionality of explanation, as with the hard problem: Assumption 2: An account of consciousness should assume the physical and explain the phenomenal. In the end, Seth argues, while phenomenal consciousness is real, it might not be what we think it is, and it should be reducible to physical processes. This closely resembles the view that Keith Frankish labels as weak illusionism [8]. Chalmers is skeptical about the power of weak illusionist theories to explain phenomenal consciousness [7]. If only physics is fundamental, it has to account for phenomenal consciousness. Under this assumption, the claims of the hard problem about the difficulty of this problem are fair, I think. If phenomenal consciousness is seen as real, but not fundamental, we have to give an account of how its phenomenal properties arise beyond function and behavior.

The Core Problem

While being more optimistic about our ability to infer phenomenal states from behavior, the real problem is arguably more parsimonious about metaphysical assumptions (the conceivability argument) than the hard problem. Could we make it even leaner in terms of its presuppositions, while leaving open the possibility for scientific inquiry? Possibly. One such idea has been put forward by Robert Chis-Ciure and Francesco Ellia called the core problem of consciousness [9]. The authors describe the hard problem as being layered in the sense that it can be decomposed (paraphrased by me, therefore possibly inaccurate):

1. There is phenomenal consciousness, which needs to be explained.
2. Conceivability arguments of non-conscious functional (and structural) equivalents are coherent. As a result, physical explanations cannot account for phenomenal consciousness.

They suggest that we can do away with the second layer, while holding on to the most basic idea of relating consciousness to physical matter. They thus define the core problem of consciousness as

We need to explain how the fact that there is something it is like to be us relates to physical matter.

It seems to me that this does away with both Assumption 1 and Assumption 2 outlined when describing the hard problem: We don’t assume any conceivability of non-conscious functional equivalents, and we don’t assign any ontological primacy to physics. In a sense, this is a much leaner version of the hard problem which still captures much of the mysteries that the hard problem addresses. The only assumptions we’re making (loosely speaking) are that both phenomenal consciousness and physics exist, and that they are somehow related. This fundamental assumption is very hard to deny, and is implied by both the real problem and the hard problem. In that sense, the core problem represents a weaker form of both the hard problem and the real problem; it thus functions as an intersection between these frameworks.

Conclusion

Combining the metaphysical leanness of the core problem with the pragmatism of the real problem (i.e. let’s assume we can at least partially measure conscious contents by behavior), we arrive at a problem statement that might be interesting to a broader range of people. Whether you’re a realist or a weak illusionist [8], the project of characterizing the relation between processes in the brain and consciousness without assuming any ontological primacy of physics or consciousness should be of interest to you. This endeavor subsumes both the scientific aspect of finding a systematic neural correlate of consciousness, as well as the question of what this relation means on a metaphysical level.

The epiphenomenal nature of consciousness suggested by the hard problem can alienate scientifically-minded people, while the reductionist prior of the real problem might turn away people who reject the assumption of physical primacy. The terminology of the core problem, on the other hand, could help scientists and philosophers, that might have some disagreements in their assumptions, find common ground. To me, an epistemologically optimistic (with respect to conscious contents, from an objective perspective) version of the core problem might be the best framework for the scientific study of phenomenal consciousness.

Bibliography

[1] Nagel, Thomas. “What is it like to be a bat?.” The philosophical review 83.4 (1974): 435–450.

[2] Van Gulick, Robert, “Consciousness”, The Stanford Encyclopedia of Philosophy (Winter 2022 Edition), Edward N. Zalta & Uri Nodelman (eds.), URL = https://plato.stanford.edu/archives/win2022/entries/consciousness/.

[3] Seth, Anil K., and Tim Bayne. “Theories of consciousness.” Nature Reviews Neuroscience (2022): 1–14.

[4] Chalmers, David J. “Facing up to the problem of consciousness.” Journal of consciousness studies 2.3 (1995): 200–219.

[5] Seth, Anil. Being you: A new science of consciousness. Penguin, 2021.

[6] Chalmers, David J. “What is a neural correlate of consciousness.” Neural correlates of consciousness: Empirical and conceptual questions (2000): 17–39.

[7] Chalmers, David. “The meta-problem of consciousness.” Journal of Consciousness Studies 25.9–10 (2018).

[8] Frankish, Keith. “Illusionism as a theory of consciousness.” Journal of Consciousness Studies 23.11–12 (2016): 11–39.

[9] Chis-Ciure, Robert, and Francesco Ellia. “Facing up to the hard problem of consciousness as an integrated information theorist.” Foundations of Science (2021): 1–17.

Originally published on Medium.