Cognitive scientist Donald Hoffman is trying to answer a big question: Do we experience the world as it really is … or as we need it to be? In this ever so slightly mind-blowing talk, he ponders how our minds construct reality for us.
I love a great mystery, and I’m fascinated by the greatest unsolved mystery in science, perhaps because it’s personal. It’s about who we are, and I can’t help but be curious.
The mystery is this: What is the relationship between your brain and your conscious experiences, such as your experience of the taste of chocolate or the feeling of velvet?
Now, this mystery is not new. In 1868, Thomas Huxley wrote, “How it is that anything so remarkable as a state of consciousness comes about as the result of irritating nervous tissue is just as unaccountable as the appearance of the genie when Aladdin rubbed his lamp.” Now, Huxley knew that brain activity and conscious experiences are correlated, but he didn’t know why. To the science of his day, it was a mystery. In the years since Huxley, science has learned a lot about brain activity, but the relationship between brain activity and conscious experiences is still a mystery. Why? Why have we made so little progress? Well, some experts think that we can’t solve this problem because we lack the necessary concepts and intelligence. We don’t expect monkeys to solve problems in quantum mechanics, and as it happens, we can’t expect our species to solve this problem either. Well, I disagree. I’m more optimistic. I think we’ve simply made a false assumption. Once we fix it, we just might solve this problem. Today, I’d like tell you what that assumption is, why it’s false, and how to fix it.
Let’s begin with a question: Do we see reality as it is? I open my eyes and I have an experience that I describe as a red tomato a meter away. As a result, I come to believe that in reality, there’s a red tomato a meter away. I then close my eyes, and my experience changes to a gray field, but is it still the case that in reality, there’s a red tomato a meter away? I think so, but could I be wrong? Could I be misinterpreting the nature of my perceptions?
We have misinterpreted our perceptions before. We used to think the Earth is flat, because it looks that way. Pythagorus discovered that we were wrong. Then we thought that the Earth is the unmoving center of the Universe, again because it looks that way. Copernicus and Galileo discovered, again, that we were wrong.
Galileo then wondered if we might be misinterpreting our experiences in other ways. He wrote: “I think that tastes, odors, colors, and so on reside in consciousness. Hence if the living creature were removed, all these qualities would be annihilated.”
Now, that’s a stunning claim. Could Galileo be right? Could we really be misinterpreting our experiences that badly? What does modern science have to say about this?
Well, neuroscientists tell us that about a third of the brain’s cortex is engaged in vision. When you simply open your eyes and look about this room, billions of neurons and trillions of synapses are engaged.
Now, this is a bit surprising, because to the extent that we think about vision at all, we think of it as like a camera. It just takes a picture of objective reality as it is. Now, there is a part of vision that’s like a camera: the eye has a lens that focuses an image on the back of the eye where there are 130 million photoreceptors, so the eye is like a 130-megapixel camera. But that doesn’t explain the billions of neurons and trillions of synapses that are engaged in vision. What are these neurons up to?
Well, neuroscientists tell us that they are creating, in real time, all the shapes, objects, colors, and motions that we see. It feels like we’re just taking a snapshot of this room the way it is, but in fact, we’re constructing everything that we see. We don’t construct the whole world at once. We construct what we need in the moment.
Now, there are many demonstrations that are quite compelling that we construct what we see. I’ll just show you two. In this example, you see some red discs with bits cut out of them, but if I just rotate the disks a little bit, suddenly, you see a 3D cube pop out of the screen. Now, the screen of course is flat, so the three-dimensional cube that you’re experiencing must be your construction.
In this next example, you see glowing blue bars with pretty sharp edges moving across a field of dots. In fact, no dots move. All I’m doing from frame to frame is changing the colors of dots from blue to black or black to blue. But when I do this quickly, your visual system creates the glowing blue bars with the sharp edges and the motion. There are many more examples, but these are just two that you construct what you see.
But neuroscientists go further. They say that we reconstruct reality. So, when I have an experience that I describe as a red tomato, that experience is actually an accurate reconstruction of the properties of a real red tomato that would exist even if I weren’t looking.
Now, why would neuroscientists say that we don’t just construct, we reconstruct? Well, the standard argument given is usually an evolutionary one. Those of our ancestors who saw more accurately had a competitive advantage compared to those who saw less accurately, and therefore they were more likely to pass on their genes. We are the offspring of those who saw more accurately, and so we can be confident that, in the normal case, our perceptions are accurate. You see this in the standard textbooks. One textbook says, for example, “Evolutionarily speaking, vision is useful precisely because it is so accurate.” So the idea is that accurate perceptions are fitter perceptions. They give you a survival advantage.
Now, is this correct? Is this the right interpretation of evolutionary theory? Well, let’s first look at a couple of examples in nature.
The Australian jewel beetle is dimpled, glossy and brown. The female is flightless. The male flies, looking, of course, for a hot female. When he finds one, he alights and mates. There’s another species in the outback, Homo sapiens. The male of this species has a massive brain that he uses to hunt for cold beer. (Laughter) And when he finds one, he drains it, and sometimes throws the bottle into the outback. Now, as it happens, these bottles are dimpled, glossy, and just the right shade of brown to tickle the fancy of these beetles. The males swarm all over the bottles trying to mate. They lose all interest in the real females. Classic case of the male leaving the female for the bottle. (Laughter) (Applause) The species almost went extinct. Australia had to change its bottles to save its beetles. (Laughter) Now, the males had successfully found females for thousands, perhaps millions of years. It looked like they saw reality as it is, but apparently not. Evolution had given them a hack. A female is anything dimpled, glossy and brown, the bigger the better. (Laughter) Even when crawling all over the bottle, the male couldn’t discover his mistake.
Now, you might say, beetles, sure, they’re very simple creatures, but surely not mammals. Mammals don’t rely on tricks. Well, I won’t dwell on this, but you get the idea. (Laughter)
So this raises an important technical question: Does natural selection really favor seeing reality as it is? Fortunately, we don’t have to wave our hands and guess; evolution is a mathematically precise theory. We can use the equations of evolution to check this out. We can have various organisms in artificial worlds compete and see which survive and which thrive, which sensory systems are more fit.
A key notion in those equations is fitness. Consider this steak: What does this steak do for the fitness of an animal? Well, for a hungry lion looking to eat, it enhances fitness. For a well-fed lion looking to mate, it doesn’t enhance fitness. And for a rabbit in any state, it doesn’t enhance fitness, so fitness does depend on reality as it is, yes, but also on the organism, its state and its action. Fitness is not the same thing as reality as it is, and it’s fitness, and not reality as it is, that figures centrally in the equations of evolution.
So, in my lab, we have run hundreds of thousands of evolutionary game simulations with lots of different randomly chosen worlds and organisms that compete for resources in those worlds. Some of the organisms see all of the reality, others see just part of the reality, and some see none of the reality, only fitness. Who wins?
Well, I hate to break it to you, but perception of reality goes extinct. In almost every simulation, organisms that see none of reality but are just tuned to fitness drive to extinction all the organisms that perceive reality as it is. So the bottom line is, evolution does not favor veridical, or accurate perceptions. Those perceptions of reality go extinct.
Now, this is a bit stunning. How can it be that not seeing the world accurately gives us a survival advantage? That is a bit counterintuitive. But remember the jewel beetle. The jewel beetle survived for thousands, perhaps millions of years, using simple tricks and hacks. What the equations of evolution are telling us is that all organisms, including us, are in the same boat as the jewel beetle. We do not see reality as it is. We’re shaped with tricks and hacks that keep us alive.
Still, we need some help with our intuitions. How can not perceiving reality as it is be useful? Well, fortunately, we have a very helpful metaphor: the desktop interface on your computer. Consider that blue icon for a TED Talk that you’re writing. Now, the icon is blue and rectangular and in the lower right corner of the desktop. Does that mean that the text file itself in the computer is blue, rectangular, and in the lower right-hand corner of the computer? Of course not. Anyone who thought that misinterprets the purpose of the interface. It’s not there to show you the reality of the computer. In fact, it’s there to hide that reality. You don’t want to know about the diodes and resistors and all the megabytes of software. If you had to deal with that, you could never write your text file or edit your photo. So the idea is that evolution has given us an interface that hides reality and guides adaptive behavior. Space and time, as you perceive them right now, are your desktop. Physical objects are simply icons in that desktop.
There’s an obvious objection. Hoffman, if you think that train coming down the track at 200 MPH is just an icon of your desktop, why don’t you step in front of it? And after you’re gone, and your theory with you, we’ll know that there’s more to that train than just an icon. Well, I wouldn’t step in front of that train for the same reason that I wouldn’t carelessly drag that icon to the trash can: not because I take the icon literally – the file is not literally blue or rectangular – but I do take it seriously. I could lose weeks of work. Similarly, evolution has shaped us with perceptual symbols that are designed to keep us alive. We’d better take them seriously. If you see a snake, don’t pick it up. If you see a cliff, don’t jump off. They’re designed to keep us safe, and we should take them seriously. That does not mean that we should take them literally. That’s a logical error.
Another objection: There’s nothing really new here. Physicists have told us for a long time that the metal of that train looks solid but really it’s mostly empty space with microscopic particles zipping around. There’s nothing new here. Well, not exactly. It’s like saying, I know that that blue icon on the desktop is not the reality of the computer, but if I pull out my trusty magnifying glass and look really closely, I see little pixels, and that’s the reality of the computer. Well, not really – you’re still on the desktop, and that’s the point. Those microscopic particles are still in space and time: they’re still in the user interface. So I’m saying something far more radical than those physicists.
Finally, you might object, look, we all see the train, therefore none of us constructs the train. But remember this example. In this example, we all see a cube, but the screen is flat, so the cube that you see is the cube that you construct. We all see a cube because we all, each one of us, constructs the cube that we see. The same is true of the train. We all see a train because we each see the train that we construct, and the same is true of all physical objects.
We’re inclined to think that perception is like a window on reality as it is. The theory of evolution is telling us that this is an incorrect interpretation of our perceptions. Instead, reality is more like a 3D desktop that’s designed to hide the complexity of the real world and guide adaptive behavior. Space as you perceive it is your desktop. Physical objects are just the icons in that desktop.
We used to think that the Earth is flat because it looks that way. Then we thought that the Earth is the unmoving center of reality because it looks that way. We were wrong. We had misinterpreted our perceptions. Now we believe that spacetime and objects are the nature of reality as it is. The theory of evolution is telling us that once again, we’re wrong. We’re misinterpreting the content of our perceptual experiences. There’s something that exists when you don’t look, but it’s not spacetime and physical objects. It’s as hard for us to let go of spacetime and objects as it is for the jewel beetle to let go of its bottle. Why? Because we’re blind to our own blindnesses. But we have an advantage over the jewel beetle: our science and technology. By peering through the lens of a telescope we discovered that the Earth is not the unmoving center of reality, and by peering through the lens of the theory of evolution we discovered that spacetime and objects are not the nature of reality. When I have a perceptual experience that I describe as a red tomato, I am interacting with reality, but that reality is not a red tomato and is nothing like a red tomato. Similarly, when I have an experience that I describe as a lion or a steak, I’m interacting with reality, but that reality is not a lion or a steak. And here’s the kicker: When I have a perceptual experience that I describe as a brain, or neurons, I am interacting with reality, but that reality is not a brain or neurons and is nothing like a brain or neurons. And that reality, whatever it is, is the real source of cause and effect in the world – not brains, not neurons. Brains and neurons have no causal powers. They cause none of our perceptual experiences, and none of our behavior. Brains and neurons are a species-specific set of symbols, a hack.
What does this mean for the mystery of consciousness? Well, it opens up new possibilities. For instance, perhaps reality is some vast machine that causes our conscious experiences. I doubt this, but it’s worth exploring. Perhaps reality is some vast, interacting network of conscious agents, simple and complex, that cause each other’s conscious experiences. Actually, this isn’t as crazy an idea as it seems, and I’m currently exploring it.
But here’s the point: Once we let go of our massively intuitive but massively false assumption about the nature of reality, it opens up new ways to think about life’s greatest mystery. I bet that reality will end up turning out to be more fascinating and unexpected than we’ve ever imagined.
The theory of evolution presents us with the ultimate dare: Dare to recognize that perception is not about seeing truth, it’s about having kids. And by the way, even this TED is just in your head.
Thank you very much.
(Applause)
Chris Anderson: If that’s really you there, thank you. So there’s so much from this. I mean, first of all, some people may just be profoundly depressed at the thought that, if evolution does not favor reality, I mean, doesn’t that to some extent undermine all our endeavors here, all our ability to think that we can think the truth, possibly even including your own theory, if you go there?
Donald Hoffman: Well, this does not stop us from a successful science. What we have is one theory that turned out to be false, that perception is like reality and reality is like our perceptions. That theory turns out to be false. Okay, throw that theory away. That doesn’t stop us from now postulating all sorts of other theories about the nature of reality, so it’s actually progress to recognize that one of our theories was false. So science continues as normal. There’s no problem here.
CA: So you think it’s possible – (Laughter) – This is cool, but what you’re saying I think is it’s possible that evolution can still get you to reason.
DH: Yes. Now that’s a very, very good point. The evolutionary game simulations that I showed were specifically about perception, and they do show that our perceptions have been shaped not to show us reality as it is, but that does not mean the same thing about our logic or mathematics. We haven’t done these simulations, but my bet is that we’ll find that there are some selection pressures for our logic and our mathematics to be at least in the direction of truth. I mean, if you’re like me, math and logic is not easy. We don’t get it all right, but at least the selection pressures are not uniformly away from true math and logic. So I think that we’ll find that we have to look at each cognitive faculty one at a time and see what evolution does to it. What’s true about perception may not be true about math and logic.
CA: I mean, really what you’re proposing is a kind of modern-day Bishop Berkeley interpretation of the world: consciousness causes matter, not the other way around.
DH: Well, it’s slightly different than Berkeley. Berkeley thought that, he was a deist, and he thought that the ultimate nature of reality is God and so forth, and I don’t need to go where Berkeley’s going, so it’s quite a bit different from Berkeley. I call this conscious realism. It’s actually a very different approach.
CA: Don, I could literally talk with you for hours, and I hope to do that.
Thanks so much for that. DH: Thank you. (Applause)