Evidence We're Living In A SIMULATION - How Evolution Hid The TRUTH | Donald Hoffman
Watch on YouTubeVideo summary
Professor Donald Hoffman, a cognitive scientist at UC Irvine since 1983, argues in this interview with Tom that human perception is not an accurate reflection of objective reality but rather a survival interface shaped by evolution. Drawing on evolutionary game theory and mathematical models developed over the last few decades, Hoffman demonstrates through simulations that natural selection favors organisms that perceive only "fitness payoffs"—information necessary for survival—rather than the true nature of their environment. He illustrates this with his famous desktop analogy: just as a computer icon does not reveal the complex circuits beneath it, our senses present us with an interface (space, time, objects) that hides the underlying reality to ensure we survive long enough to reproduce. Consequently, believing in space-time and physical objects is mathematically equivalent to being wrong about objective truth; evolution would never favor a creature capable of seeing reality as it truly is because such perception is too complex and irrelevant for survival compared to fitness-based shortcuts. The core mathematical proof relies on the concept that if an organism sees reality correctly, its probability of surviving decreases significantly compared to one tuned only to payoffs like food or mates. Hoffman explains this using game theory where strategies are defined by their reproductive success rather than truthfulness; a cheater who hides while others work might survive better in certain scenarios, but too many cheats collapse the system. His simulations show that evolved creatures develop perceptions completely divorced from reality, seeing only what helps them acquire fitness points. This leads to his "consciousness all the way down" hypothesis and the "kid in the candy store theory," where consciousness is fundamental rather than emergent from brain activity. In this view, neurons are not biological hardware generating mind but symbols within a social network of conscious agents interacting on an infinite landscape of mathematical structures described by Gödel's incompleteness theorem. Hoffman addresses the nature of self and identity through split-brain patients, suggesting that what we perceive as one unified consciousness is actually a lattice of countless interacting conscious agents at various levels of complexity. He posits that beneath our "headset" lies not unconscious matter but pure awareness without experience or content, a state akin to mystical traditions where the sense of self dissolves. While he remains skeptical about using psychedelics due to personal paranoia and potential side effects like terror rather than bliss, he acknowledges they might offer temporary glimpses into this deeper reality. The ultimate goal is not merely to understand our current simulation but to reverse-engineer it; by understanding how conscious agents project themselves onto space-time via scattering amplitudes in quantum mechanics, humanity could eventually gain access to the "source code" of existence, allowing us to manipulate or exit the constraints of spacetime entirely. The interview concludes with a reflection on the profound implications of this worldview for human experience and future exploration. Hoffman compares the process of expanding consciousness to a caterpillar undergoing metamorphosis into a butterfly—a painful dissolution of current structures that allows access to new possibilities. He notes that most deeply held beliefs, including those in flat earth or fundamental spacetime physics, are likely incorrect because they represent outdated interfaces rather than truth. The conversation highlights how our interactions with others and loved ones occur within this headset, providing meaning without revealing the underlying mechanics. Ultimately, Hoffman invites listeners to consider that stepping outside current perceptions might involve losing one's sense of self entirely, yet it offers a path toward exploring an infinite variety of conscious experiences beyond the limitations of biology and physics as currently understood.
Read the full video transcript
[Music]
What is up everybody? Welcome to another
episode of Conversations with Tom. I am
here with somebody I think you guys are
going to find deeply fascinating, Donald
Hoffman. Um, Donald, welcome to the
show.
Thank you very much, Tom. It's a
pleasure to be here. Thanks for inviting
me.
Of course, man. Dude, after so I did an
interview with Anukica Harris. She
introduced me to some of the concepts
that we'll be talking about today, which
are uh one crazier than the next, but
super interesting. But before we dive
into that, give people a super quick
synopsis of who you are, your
background, all that good stuff.
Uh, I'm a professor at the University of
California at Irvine in the cognitive
science department. I've been there
since 1983. And, uh, so I study visual
perception,
um, perception of objects and their
their colors and their shapes and their
motions. And then for the last uh 20 30
years I've been studying consciousness
as well and trying to develop a
mathematical model of consciousness and
then studying evolution and asking does
evolution shape our senses to see
reality as it is or not. So those are
sort of the big topics I've been
studying.
Yeah, it a lot of this seems to go back
to vision and the way that we literally
see things. And by the way, don't and I
don't know how much you know about me or
my obsessions, but I'm obsessed with the
movie The Matrix.
Yes.
And in your dedication, you I forget the
names of the three people, but you said
I offer you the red pill,
which I thought was pretty interesting.
So,
what in your world, what is the red
pill? What is the red pill waking us up
to?
Right. Most of us believe that we see
reality as it is. When we look up and
see the moon, it's because there really
is a moon. And it would exist even if
there were no observers to see the moon.
It would still exist. And we don't
believe that we see all of reality. No
one thinks that we see everything that
there is to see. But we do believe that
we've been shaped by natural selection
to see those aspects of the truth that
we need to see to stay alive. And so
that our perceptions of space and time
are giving us a genuine insight into a
real space and time that would be there
even if there were no observers to
perceive it. And also our perception of
objects like tables and chairs, the
moon, quarks and leptons and so forth
that these things would also exist and
have roughly the properties that we see
even if there were no creatures, no
observers to see them at all. And the
red pill that I'm offering is to say
that if you believe evolution by natural
selection then the mathematics of
natural selection makes it very very
clear that the probability is zero that
any of the language that we use in our
perceptions the language of space and
time the language of shapes and objects
and position and momentum and colors and
so forth is the wrong language to
describe objective reality whatever that
reality might be. It's not that we're
getting the shape of this table a little
bit wrong or the colors a little bit
off. It's that no description in the
language of space and time and objects
and colors could ever be be true.
Reality, whatever it is, can't be
described in that language if we buy
evolution by natural selection. So we
have a choice between taking one of our
best confirmed scientific theories
seriously, namely evolution by natural
selection, or taking our intuitions that
spacetime is fundamental and objects are
fundamental, taking that intuition
seriously. And I decide to to side with
this science on this one.
All right. So this a lot of this stuff
is going to be really difficult to tease
out, but I think we have to sort of
break it into points. I don't want to
spend our whole time um describing the
theory. We may get lost for a fair
amount of time in that,
but I what I hope we can do is sort of
lay out the thinking and then get into
some of the implications, why it matters
and all that good stuff. Um, so
if
I guess let's back up. So
tell me how natural selection gives us
the mathematical model that invalidates
all that. That's probably the right
place to start,
right? Most of us think of natural
selection and evolution as a biological
theory that that's that's sort of
intuitive, right? you know, if you are
better adapted to the environment,
you're more likely to pass on your genes
and so forth. And but it it turns out
that in the 1970s, uh, a brilliant guy
named John Maynard Smith was able to use
the ideas of evolution by natural
selection and the tools of mathematics
in spec particular, uh, game theory to
create a new field, evolutionary game
theory.
Can you can you explain game theory? So
the when I was diving into your world
when I was reading the book when you
talk about miracles and I don't want to
do this out of order but you talk about
like every theory has sort of a base
assumption and it's like we can't we
can't explain that one so just please
accept that this is the the miracle as
you call it and then from here the rest
of my argument is going to make sense
because I don't understand the math well
enough. The math every time you mention
this and nobody pushes you on it.
I don't understand the math well enough
it feels like the miracle to me but it
isn't to you. So what actually what is
the math if we can sort of say it at a
at a lay person's level? What is the
math? How does it show that there's a
zero probability that we're describing
reality as it actually exists?
Right? So so the intuition behind using
game theory for evolution is very much
like if you play a video game, right? If
you're playing a video game, you have to
focus on getting points as fast as you
can. If you get enough in a short enough
time, you get to the next level,
otherwise you die. And an evolution is
very much like that. There's points,
they're called fitness payoffs, and you
have to gather them as quickly as you
can. And if you get enough roughly in a
short enough period of time, you don't
go to the next level, but your genes in
your children go to the next level. And
so it's that's roughly why we want to
think about game theoretic kinds of
things. But it's of course it's more
detailed than what I
game theory is literally that what
survives to the next round.
Yeah. So game theory is literally a
mathematical theory of strategies in
games and how different strategies may
be better or worse in certain
circumstances. So for example, if if you
one game that our species plays is a
social cooperation game. And we were
hunter gatherers. If we all went out and
cooperated, I went out and hunted
wilderbeasts and you went out and hunted
wilderbeasts and others gathered berries
and so forth. We all worked hard and
came back at the end of the day. If I
didn't get enough and you got more, then
you might share with me and tomorrow I
might share with you. So that's
cooperation. But and so that works very
very well. That's that's a strategy that
works very well.
How would we define this game? So I was
tempted to say, okay, so it's about the
people who have the strategy to win this
game called life. But I'm not sure
that's actually how you define it. Would
would it be the game called procreation
or
Okay. So in evolution,
fitness is all all about procreation. In
fact, basically that's how it's defined.
It's almost um it almost sounds like
it's a tutology that fitness being more
fit means that you're having more
offspring. So whatever you do that lets
you have more offspring is by definition
giving you greater fitness. And so so in
the case of cooperation, I mean that's
not the only strategy by which a species
could be successful at having offspring
in the next generation. But for homo
sapiens, that was a strategy that we did
use. And and many social species do
that. Ants and bees and so forth. But as
soon as you have that strategy of
cooperation, then it turns out there's
another strategy that could be very fit,
and that's cheating. So I could pretend
I'm I'm a loafer now. I don't want to go
out there and put my life on the line in
front of a wilderbeast. And so I just go
down to the river. I hide out and relax
and take it easy and come back and go,
"Oh, I worked really hard today. I
couldn't find and think I share some of
yours."
Well, and and so it turns out that that
strategy is very very fit in the sense
that I didn't put my life on the line.
If you're willing to share with me, I'm
going to survive. So I'm going to have
So that strategy will actually
proliferate. I'll have kids. I'll have
more loafers. And so, but it turns out
if you if have too many loafers, too
many cheaters, then no one's bringing
home the bacon
and the whole thing collapses. And so
you get the idea that the the the the
fitness of a strategy depends on the
other strategies that are around. If
everybody else is cooperating, then
being a cheater is very very effective.
It's a it's a fit strategy. If
everybody's a cheater, then it's not fit
to be a cheater because everybody's
going to die. And so that's what we call
frequency dependent selection. that the
frequency of a different of a strategy
um will affect its fitness. And so that
makes it more complicated and that's why
we have to do mathematics. It's you can
actually with mathematical precision
predict exactly what proportion of the
population will be cheaters and what
proportion will be cooperators once you
know certain things about the strategies
and and their payoffs. And so it's
that's why we use evolutionary game
theory. allows us to go away from just
intuitive
notions of evolution and selection and
so forth to precise definitions of
strategies of their fitness, how they
interact. When h when three or four or
five or n strategies interact, what
happens? It gets very very complicated.
Then our intuitions just give up. But
the math can still carry on where our
intuitions give up.
Okay. So you partner with a
mathematician and did you have a theory
that you wanted to see if he could write
the the sort of algorithm for or how did
that work? How did we end up focusing on
math?
Right. So
in about 2008 I decided I really wanted
to go after this because I I had a
suspicion that evolution would not favor
creatures that see reality as it is. I
figured that it would be too complicated
and take too much time. And it turned
out that was correct, but but it was
more interesting. So I got a couple of
my graduate students working on this. I
worked with them. We learned
evolutionary game theory and we started
they they wrote simulations. And so we
just simulated creatures with different
strategies and we let them see all of
the reality or none of the reality. We
let them just see fitness payoffs.
Give me some of the data points. So one
of them is going to be fitness payoffs,
right? when you're writing the math, um,
like are you making this from a human
perspective? Are you taking this from an
ants perspective? I've got to imagine
the math looks differently depending on
the species strategy that's already
evolved sort of over evolutionary time.
So, oh god, I can only imagine how
complicated this is, but give me a give
me a couple of the variables so that
this stops being one of the miracles for
me and I can really start to understand.
Sure. So, one kind of game that we had
them play was a foraging game. So you
could think of big a big checkerboard or
a big chessboard
and you had them play. What do you mean
them?
The So so my graduate students then had
these simulated creatures and we just
plop them down at random in this big
huge checkerboard.
Okay.
And we had also planted resources around
on the checkerboard.
And did you have to give them
incentives?
Well, in in some cases we would just we
would either allow them to evolve
incentives. So that was what we called a
genetic algorithm. So we started off and
they were all stupid. They they all um
had random genes for how they moved and
how they would try to feed and how they
would perceive.
But there had to be some sense of score,
right? So if we're using game theory,
there has to be some sense of you have
an objective, that objective is to
gobble the resources or whatever.
Yeah. So certain resources that were
just randomly distributed on the
checkerboard gave you high fitness
payoffs, some gave you low fitness
payoffs and and so forth. So we just
place
So it's just defined as fitness payoff
without anything beyond that. Exactly.
They had they had a a a driver, if you
will, to put it in human terms, they had
a drive to acquire fitness payoffs.
Right. Right. Got it. Um, at least in
when we when we programmed them now,
when we just had the genetic algorithm,
there was no drive at all. They they
they just
what would make them move?
So, they they had to do something, but
the genes were random. So, they they
would do stupid stuff like they would
stay in one spot and try to eat and and
they would get nothing and they just
keep trying to eat there the whole time
until that whole and they would just die
off, right? So in the first generation,
right, we
did they have the desire to eat and
procreate? Like were they looking to
mate or how did you
all the only the only desire was we we
told them they had to do something every
step of the game? That's all we did.
And was there a list of items for them
to choose from? Eat, move.
Yeah. Yeah, that's right. Eat and move.
And that was pretty much it. And and
step. Yeah, eat and move. That's right.
And then you had something on the back
end that said if they score this many
number of points then they essentially
procreate and they move on to the next
generation. If they don't they're dead.
That's exactly.
Okay. Got it.
So the first generation pretty much all
of them just acted very stupidly. They
didn't get any fitness points or very
very few. Some bumped against walls and
just kept bumping into the walls. And so
we would just breed the ones that were a
little bit less stupid than the rest
and then make a new generation of them.
And we did that for four or 500
generations. And then by the time we'd
done that 500 generations, we had
creatures that looked very purposeful.
They were foraging in an almost optimal
p pattern. They weren't wasting any time
and they were getting high fitness
payoffs. And we could look at their
perceptions.
And we found that they evolved
perceptions that didn't show them the
truth. It showed them only fitness
payoffs, which was no surprise to me.
But but what was a surprise was the
reason why. It wasn't just that it was
too expensive to to see the truth. It
was that
seeing the truth
and seeing what you need to survive are
very very different things.
And and so so I went to a mathematicians
on once I had the simulations and I
realized that this looked like it was a
real result and and I gotten some new
ideas about what was really going on. So
the simulations really taught me it
wasn't just that it was too time and and
resource expensive to see the truth. It
was that in some sense the truth is
completely irrelevant that that it's and
and also that the fitness payoffs
themselves don't tell you anything about
the truth. They're they're just
independent.
So I went and and worked with a
mathematician named Chayon Pash and um
we've worked together. He's the
mathematician. I'm I'm not. And we have
and I've gotten some other collaborators
as well, some very very good
collaborators. And we have a couple
papers that we've now written where with
two different angles on on the theorem.
And the bottom line is is this. Um it's
it's straightforward to prove that well
straightforward if you have a
mathematician working with you. Um but
to prove that the an organism that sees
reality as it is in whole or in part is
never more fit than an organism of equal
complexity that sees none of reality
whose senses actually don't have the
right language to see reality and is
just instead tuned to the fitness
payoffs.
Okay, so this is sort of the first thing
that I trip over. So, um, I get where I
could see that it is more advantageous
to be optimized for fitness payoffs than
it is for reality. And this might be a
good time to give people your sort of VR
explanation so that we we can bring this
into something they can visualize. But
first, let me finish what I'm bumping
on. So,
I get how if you're optimized for
fitness payoffs, that makes more sense
than being optimized for reality.
Reality could have too much just
complexity. The processing power that it
would take to understand is crazy. which
is exactly why I think your desktop
analogy is probably the better one to
hit right now. Um,
but what I don't understand is why it is
necessarily true that you need to hide
reality like why that would be part of
it. That seems
sort of a bridge too far for my my
simplistic mind.
So that is a bit technical but I but the
top level idea is um that fitness
payoffs depend on the state of the
world. Right? So to be concrete
and the organism, right? organism,
right? They depend on the world and the
organism and the state of the organism
and the action. So, I mean, one example
is, you know, if I have a t-bone steak
and you know, the fitness payoff of that
t-bone steak for a hungry lion is pretty
high. But for the same lion that wants
to mate, it's very, very low. And for,
you know, a cow in any state and for any
action, the T-bone steak offers no
fitness payoffs. And and another example
is if I'm 5,000 meters underwater,
that's pretty bad for me. But for a
benthic fish, that's it's perfect for a
benthic fish, right? So so evolution by
natural selection has this idea that
there is an objective reality. And
fitness payoffs do depend on that
reality. But the payoff for the same
state of reality could be very very
different for a benthic fish than for
me. For a benthic fish, 5,000 meters
underwater is the same state of reality
as it would be for me. But the payoff is
very very different for the benthic
fish. It would kill me. this this may
not be the thing to dive into, but I'm
going to push on this a little bit and
see if we get somewhere fruitful. Um,
what where I sort of come to in your
theory is that there lurking under this
is is a reality, right? The moon is
there to describe something. It's my
shorthand. It's not It is not a
meaningless shortorthhand.
It is a shortand to help me get my um my
fitness payoffs fair, but it it
represents something,
right?
Okay. So if that is true,
understanding that I am 5,000 ft
underwater still seems relevant even
though it's not it doesn't have a
fitness payoff. So obscuring that or
obfuscating that for me so that I can't
understand that I'm 5,000 ft underwater
is not helpful. Okay,
I can tell that if we chase that, we
won't get where we want to go. But if
you give us the desktop analogy, right,
because this I think will will give us
the anchor that we need to keep
exploring,
right?
So, so if evolution didn't shape us to
see the truth, what did it shape us to
see?
And the I think the the good analogy is
that it gave us like a desktop
interface. So, if you're um writing an
email and the icon for the email you're
writing is blue and rectangular in the
middle of your screen, does that mean
the email in your computer is blue
rectangular in the middle of the screen?
Middle of the computer? Of course not.
That I mean, anybody who thought that
misunderstands the point of the
interface. is there not to show you the
truth, which in this metaphor would be
the circuits and software, the diodes
and resistors, magnetic fields. You
don't want to deal with that. If you if
you had to deal with magnetic fields to
write an email, good luck. You would
never no one would hear from you. And so
that's what evolution did for us. It
gave us a desktop interface that's there
to hide the truth, right? The desktop
interface on your computer is there
explicitly to hide the circuits and
software. You don't want to see that
stuff. That would seeing the truth would
get in the way. But isn't evolution
somewhat of a blind watchmaker? I mean
to just steal from from my man Mr.
Dawkins. So if it is blind, it's not
hiding anything. So it's just it's
optimizing you for something. So that's
where I get into the
like
the the punchline of your theory, not to
get too far ahead, but for people to
understand why I'm stopping you. So the
the punch line of the theory is we are
so the far like so far off from what is
real as to like not even be able to
to conceive of what our world really is.
And we will definitely get into spaceime
is doomed. It's one of the most
fascinating things to come out of um
your theory. But it's like
I it doesn't feel like a blind watch
maker is going to hide something from
me. It it just it it has no sense of it.
It's just here is the shortest path
across the checkerboard to get to this
thing. It's not trying to trick me into
thinking that there is no checkerboard.
It's just like take this path, go that
way. It's the shortest path. I'm
essentially lazy, right? If you think
about um caloric realities, right? So,
the way that I look at a a human um I I
want to write a book called The Physics
of Being Human.
And to do that, I would really it's the
like you write the book that you need.
So, to understand in myself why I am
both driven and lazy is so [ __ ]
weird. So, I both have these huge
ambitions. I want to do this crazy [ __ ]
but I also want to sit on the couch and
eat chips, right?
So, it's like, and they're both real,
man. I am not like when people think
that like I'm being humble or whatever.
No, no, no. I really have like a
hardcore drive to sit around and do
nothing,
but I have this like sort of weird
balancing thing. So, for me to
understand it from a a caloric
utilization standpoint,
right,
as humans, we have these massive brains
that are calorically just ravenous. And
so for me not to have to forage all the
time, I take a strategy where I'm
conserving calories. Again, blind
watchmaker, not not somebody going, hey,
you know, it'd be really smart. It just
realized the ones that didn't conserve
calories [ __ ] died. And you know,
they didn't survive a famine or
whatever. And the ones that sort of
balanced this like, I'm going to go down
and I'm going to face the will of the
beast and I'm not going to hide all the
time. You know, I'm going to put myself
at risk and and all that stuff. The the
ones that found that balance, they were
the ones that procreate and they pass on
their genes. Okay. So part of the
physics of being human is to both be
adventurous and lazy at the same time.
Where where I'm trying to like figure
all this out then is okay, I have this
layer. I have my interface, but it
doesn't feel like it's that like
radically far from the the truth. So
I'll I'll give you my example. So all
right, it's mapping this room. You're
going to tell me, "Fuck, dude, you don't
even understand. Like spaceime isn't
real. None of this [ __ ] is real." I look
away, the moon doesn't even exist.
Right. Right.
That seems
like it would be more problematic for me
to navigate the world if all of that
were true.
So, what I want to understand is how
much of of what you say about the
computer interface is really that
divorced from reality,
right?
Oh, man. Stick with me. I'm going to see
if I can actually articulate this.
So, the the computer analogy
is super profound. It's easy for me to
understand. I don't want to have to deal
with electrical fields and
all the other things, the diodes or
whatever, all that stuff.
But when I think about whacking into a
rock, that seems like a way closer thing
to my reality. I've mapped something
over that.
Same with the moon. I've mapped
something over it. It isn't actually
that thing,
right?
But it feels like it would be mapping
over some gravitational object that
rotates around the Earth. But I think
you're going to say that's not true.
Right. So,
what analogy because it our intuitions
rebel at the idea we're not seeing the
truth because it's what we're doing here
works so well. Right. And
I'm open to that. We're not seeing the
truth. Where where I get lost is how
[ __ ] divorced can it really be and
still be useful.
Right. Well, so there's two aspects to
that. One one is just that you know the
mathematics of evolution is quite clear.
Meaning the probability that you see the
truth is zero.
Is zero. That's right. And and that we
have a paper that we just submitted on
Monday um where we we show that the
fitness payoffs erase all evidence of
world structure almost surely with
what the [ __ ] does that mean?
When you say world structure, what do
you mean?
So the world presumably there is some
world some reality.
Yes. Like atoms.
Uh well, you don't need to postulate
exactly what that word.
I need to know what you mean by the word
the word structure. So it could be and
what we show is it doesn't matter what
the structure is pretty much the result
holds. So it could be for example a
structure like uh a distance
relationship a metric what we call a
metric or a topology or a measurable
structure okay
or a total order like one is smaller
than two is smaller than three that
that's a total order right
so whatever the so what what you can
show is that no matter what structure
you might think the world has you can
prove that the
fitness payoff functions that govern our
evolution so you're right there's no of
course goal right it's not a
goal-directed kind of thing, right? You
know, evolution is not trying to do
anything. And but the point is that the
fitness payoffs which govern evolution,
the probability that they will actually
preserve the structure in the world. So
that by being tuned to the fitness,
you're tuned to the structure in the
world.
Yeah,
that probability is precisely zero.
That's what we prove.
So the only thing that we know is what
you think is real is the only thing we
know is not real.
Is that accurate?
Well, I would say according to theory of
evolution. Now, we can step back and ask
ourselves what the theory of, you know,
what we think about the theory of
evolution, right?
But before we get to that, because you
you do you have a pretty deaf challenge
to how cuz my first literally the first
note I took about you was my whole
problem is you're laying out, hey,
evolution mathematically proves that
essentially evolution isn't true. So, I
was like, if if every I think evolution
is true, how can it also be true that
everything I know and think is false?
Right? which is essentially what I just
heard you say. So mathematically I can
prove this the quote unquote structure
of everything I think is absolutely not
true
and yet I'm using evolution to base that
math on.
Right.
Walk people through how that isn't a
contradiction.
Right. So it's it's what we show is that
the fitness payoff functions
erase any information about the
structure of the world. So that the
structure of our perceptions is just
unrelated to the structure in the
objective world. But the argument that I
just gave does not apply to our math and
logic. So
so just perception
just perceptions. So you have to be very
very we have to look at all of our
cognitive capacities one.
Is this whole thing anvelt problem?
Well, it what it's saying is that uh
your envelope
that every creature, you know, homo
sapiens has onevel, one user interface.
We have the Apple interface and someone
else, you know, some other creatures
have the Mac or the PC interface or or
whatever.
Y
and and and there's going to be a wide
variety of interfaces that evolution
evolves. Every species has its own class
of interfaces. Um and in each case the
interface never shows any species the
truth at all according to evolution. But
there are selection the selection
pressures that erase information about
the structure of the world in perception
do not also apply to math and logic. The
reason is that we do have to have some
elementary ability to reason about
fitness payoffs. Two bites of an apple
give me roughly twice the fitness
payoffs of one. So not reasoning about
objective reality, just reasoning about
fitness payoffs and the logic of fitness
payoffs. Right? So that's why there's no
selection pressures necessarily to be
geniuses at math and logic. But at least
the selection pressures are not
uniformly against any capacity in math
and logic. Whereas in the case of
perception it's one can show the uni the
pressures are uniformly against any
access to the structure of the world um
in in terms of the structure of what we
perceive in in our senses. So that's why
we have to be very very careful. Certain
for example Christian philosophers Alvin
Plantinga for example has argued from
not mathematically but informally from
evolution saying that it would make all
of our cognitive capacities unreliable
and therefore evolution by itself was
unreliable theory and therefore we
should not you know not believe it and
I'm not saying anything like that at
all. is saying that the theory of
evolution
has a core that John Maynard Smith found
evolutionary game theory when we look at
that core we find that there are certain
peripheral assumptions like DNA exists
whether or not it's perceived space and
time exist those peripheral assumptions
turn out to contradict the mathematical
core of the theory and it's you one can
prove that but math and logic our
ability with math and logic does not
contradict the evolutionary the core of
evolutionary theory so is we have to be
very very careful. That's why you when
you do this you know it's it's not just
handwave anymore. You really have to
look at the replicator equation. You
really have to look at the fitness
payoff functions and do cominatorial
analyses and so forth. This is very very
careful work. But that's what we do with
our best scientific theories. We take
them very very seriously. We look at
their equations and say okay if the
equation entails
the probability zero that we see reality
as it is then we've we've got a choice.
We can agree that we don't see reality
as it is or we can say we need to revise
the theory. Now, we don't have an
alternative to evolution by natural
selection. So, if someone wants to
propose one, they've got a lot of work
to do because evolution by natural
selection is an incredibly successful
theory.
So, spacetime though and you're you're
ready to ditch that one. Sure.
So, um okay. So I'm I'm getting I think
what you're saying about p our unvelt
and uh because we haven't defined that
I'll define that. So uh is your senses
take in the world in a certain way and
uh we have different senses in a bat. So
a bat can do echolocation. We cannot and
therefore the way that a bat interprets
the world is very different than the way
that we interpret the world. And every
sort of species has a different enovelt.
You can even have humans that have a
diff a slightly different oovelt. One
that's colorb blind. Um there's a
million other examples, but you you get
these variations that are massive
between species. Um you get like a dog
the the amount that they can smell is
crazy. They can smell a seizure coming,
which is absolutely bananas, whereas of
course you're not going to get a human
that does that because of the number of
um scent receptors in the nasal canal.
And so cool. All right. that that's an
unvelt. So I get perception. I get that
this is a a problem of perception.
I think I understand the math part. I
pro because I'm going to I am making a
layman's assumption that math is a
universal language. I've heard that
repeated a million times. I am so bad at
math. I don't even understand that. Uh
but I accept it out of ignorance. uh the
logic part. Let me define logic based on
what I think you're saying
and you tell me if I'm getting it
correct. So logic the way that I
understand it from the way that you just
use it because I would have said it's
human reasoning. So I would have gotten
tripped up um on like natural selection
or oh god the one you just used it as an
example and you said that that we think
spacetime a great example right we have
in my layman's view we have logicked our
way to spacetime
but that's failed me so
um
tying it back to fitness payoffs I think
is is your definition of logic that we
have to be able to reason that one bite
of an apple is not as good as two bites
of an apple. Is it really that basic
that it's it is entirely tied to fitness
payoffs?
That's well from an from the
evolutionary arguments that I'm giving.
Right. So the arguments that that
but do you think that logic reaches
deeper than that when you say that it's
untouched by this um false
interpretation. Oh god I'm putting words
in your mouth.
Well, right. So so that gets to the
bigger picture of what I'm up to here.
Right. So and this is how science
progresses. What we do um is we take our
current best theories and we try to push
them to their limits and find out where
they break,
where they fall apart. And when we do
that, that's when we break out the
champagne. Because the whole point in
science is to push our best theories to
the limit to find out where they break
down and then get some clue about a
deeper theoretical framework. And the
constraint on that deeper theoretical
framework is it better agree with our
current theory where our current theory
is right.
But
only if you're saying that that theory
is logic. This is another one of those
times where I bumped against what you're
saying. So and and this may be that
simple. Is that what you mean? That like
where we are using true logic
as you define it tied to fitness payoffs
I think.
Um then it must agree because you've
said if you work backwards I'm not
trying to get rid of evolution. I'm not
trying to get rid of the things that we
know like we can launch a satellite into
space and geo target you but that
requires relativity. So whatever we um
get to whatever answer we come to better
be backwards compatible with the ideas
that essentially work.
Can you give me a definition though of
work and is it tied to logic as you
define it and thusly fitness payoffs.
Right. So, so the idea would be that
scientists are doing reasoning, right?
Science is about reasoning.
Is reasoning a synonym for logic to you?
Pretty much. That's right. And if we um
we can't
logic is in some sense non-negotiable in
the sense that if I let go of reason and
logic then there's nothing left. Right?
Is that true?
Because all we can do if we if we're
having a conversation
Mhm.
we can talk informally but we can't make
very good progress unless we are
absolutely precise in what we say.
That but let me tell you how that struck
me. So does somebody who's schizophrenic
do they have reason and logic? Well,
in the sense that if we're trying to
understand our situation in the
universe,
humans will make stories. Yes,
if the stories are internally
self-contradicting,
you can be pretty sure you're going to
be in trouble, right? So, any internal
contradictions are going to destroy your
theory. So the first thing you have to
do is make sure that what you're saying
doesn't contradict what you're saying.
If you because that just means you're
saying nonsense.
So one one reason we we use math and
logic is to make sure that what we're
saying isn't just flatout nonsense. But
once we're past that criterion and and
much of what we've said is nonsense, but
once we get past that criterion, then it
turns out we may have been using our
terms very imprecisely. We might use the
word space. We might use the word time.
and we think we know what we're talking
about, but when you actually push, you
find out, oh no, I I would say this in
this situation, that in that situation.
Oh, and they contradict. I've
contradicted myself again. So, once
again, we find that if we're we just use
intuitive concepts, we get trapped in
self-contradiction, namely nonsense. So,
the whole point of being mathematically
precise, one point is to make sure that
we're not doing nonsense. The second is
to take our ideas and force us to put
our ideas very very precisely so we know
exactly what we're saying.
I get the math part though, the part
that I'm really trying to wrap my head
around and maybe we should just move the
[ __ ] off this, but I'll take one more
swing at it. Um,
math I get.
If if math truly is a universal language
that just sort of gets at the substrate
of what this whatever this is, it
actually gets to that. Cool. I can see
how um the fact that we that uh chasing
fitness payoffs manipulates our
envelopics
think they're being logical, right?
What on earth makes us think that we are
not? just the way that our our um
reality is essentially this total
abstraction, right, in your own theory
and and I would agree, man. I I
routinely think in myself and and
express other people. Your brain is
creating a virtual reality. Now, I never
ever ever conceived of it as sort of
radically different as you, but um I get
that we're in sort of this this huge
abstraction, but what makes us think
that that our the way that we reason,
the way that we logic outside of math,
the way that we reason and logic outside
of math isn't the same as a
schizophrenic where it is it is so
delusional. Is it just the internal
consistency or and couldn't that itself
be a delusion that our fear of internal
um contradiction is problematic? Is is
actually a delusion in and of itself?
Oh, very very good. So I see your
question now. So yes, there are deep
issues here in terms of of logic and
there is something called Girdle's
incompleteness theorem that basically
Give it to me.
You familiar with that?
No, not at all.
So So Girdle is probably
arguably one of the most profound
results ever in human thought. Girdle
proved it's called Girdle's
incompleteness theorem and he proved
that any axiomatic logical or
mathematical system that's rich enough
to do arithmetic um there will be
statements that are true but can't be
proven within that system unprovable
truths the notion of truth
goes deeper than the notion of proof in
that system you might say well okay I'll
take that truth and put it into my
axioms then he said well but then
there'll be new truths that you can't
prove and this goes on forever what he
showed was that the exploration of
mathematical structure is in principle
endless. It's unbounded. And in fact, if
we get get to it later on, my theory of
consciousness, I'll argue that that's
what consciousness is up to. This
unbounded exploration of all the
possibilities of consciousness that
comes from Girdle's theorem. But you're
you're absolutely right. We can choose.
Give me Girdle's theorem one more time.
Right. Right.
As simply as you can.
The the simple bottom line is there is
no end
to the exploration of mathematical
structure.
It's turtles all the way down.
All the way. And in principle, you can
never know at all.
Okay. Interesting. And I definitely get
how this ties into your theory of
consciousness. So, let's spend a minute
on turtles all the way down. Uh super
cool story. It I I don't remember where
this started, so if you do, by all
means, jump in. But it uh woman was
claiming that the earth is flat. And so
somebody said, "Okay, if it's flat, then
what is it sitting on?" And the story
goes that it's sitting on a giant
turtle. And so the person asked her,
"What is the turtle sitting on?" And she
said, "Don't be ridiculous. It's turtles
all the way down." And of course, you
get into this. Eventually, one of those
turtles is on something. So, and that's
that same human like visceral reaction
is how I react to your theory of
consciousness. So,
why don't you give it to us and then
we're going to start playing around with
the brain.
Okay.
Um, but what is your theory of
consciousness?
Right. So,
I'm saying that space and time aren't
fundamental
because evolution by natural selection
entails that.
Not only not fundamental, but not real.
Right.
That's right. They're they're only real
as forms of our experience. They're not
That's right. They're our own. They're
not real.
They're part of the virtual reality
desktop abstraction.
That's right.
Okay.
But they're not real in the sense of an
objective reality in the sense that they
would be there even if no creature were
around to perceive them. So it's again
like a virtual reality. Every time like
if you're playing a a game of uh you
know fast race cars in in virtual
reality, I see a red Corvette.
Sorry. I'm so sorry to interrupt, but uh
would another way to say that be because
you say when I look at the moon and then
I look away that I you haven't said this
in this interview, but I've heard you
say it many times that you you trash bin
it or something like that. You're no
longer rendering it as garbage
collector.
Garbage collector.
Um would another way to say that the
moon doesn't exist quote unquote if
there were no species to perceive it.
It's really that there the moon wouldn't
be um created essentially. It's like it
it it is requiring an oovelt of a
particular species to create that
shortorthhand over evolution to say
whatever that thing is that the moon is
meant to represent.
Um it has been created by humans as a
moon. Maybe bats see it entirely
differently and fish don't have any
sense that it's real at all. Who knows?
Um it's really that we
we evolution maybe is is a better way to
say it. that evolution has created this
virtual reality and therefore if that
species ceased to exist, its version of
virtual reality would obviously cease to
exist because it's being created.
Exactly.
Oh, but I'm going to say it's being
created by the brain. You're going to
say the brain's not real.
Uh is that true? Right. The brain. Okay.
Yeah. This is where the [ __ ] gets so
complicated. Okay. So, I'm going to shut
up now. Uh back to your theory of
consciousness.
Right. So, but I just point your your VR
thing is is I think a a good example for
anybody who's spent time in VR. What I'm
saying will be obvious, right? If you're
playing a VR game of like race cars, you
see a red Corvette. When you turn your
headset that way, you know that you're
only seeing a Corvette that you're
creating when you turn your head that
way. You turn your head to the other
side, now you're seeing a blue Mustang.
The red Corvette is gone. It doesn't
exist. There's no red Corvette in the
computer that's running the game. The
red Corvette is only in your mind when
you look over there. Now you're seeing a
blue Mustang because you're making that.
And so you you're rendering these things
and then destroying them. There is a
reality, but it's not corvettes and it's
not Mustangs. It's the supercomput
that's running the game. And that's what
I'm all I'm saying is evolution gave us
this headset and it's no surprise. I see
the moon, I render a moon. I turn away,
I don't render a moon. So, the moon
doesn't exist. There is something, but
it's just not like it's it's not the
moon. It's nothing like the moon. Just
like there's something there's a super
supercomput in the the VR analogy, but
in the supercomputer, if you looked,
you'll never find any, you know, green
Mustangs or or red Corvettes. You give
an example in your book that is so
powerful. If you would take a second,
you describe what's happening in your
eye when you look at a scene that
includes a red apple. And the way you
describe it at the photo receptor level,
I was like, "Oh my god, it gave me such
an understanding of how terrifyingly
complicated things actually." Do you
remember the part that I'm talking
about?
Right. So, this is now just normal
physiology. And so for the moment I'll
be talking as though you know I believe
in brain science and and neuro and and
and we have to do that right
if right so we have to bracket
everything is within the framework of a
theory so I'm now using neurohysiology
and and physics right now for this to
describe this
so when you look at a red apple and
suppose there really is a red apple just
for sake of this argument it's got a
real shape and light rays hit it and
they have certain frequencies and they
pass through the lens of your eye which
focuses it on the back of your eye just
like like a camera would. And on the
back of your eye, you've got a piece of
brain called the retina. It's a it's
nervous tissue. So, it's a piece of
nervous tissue. It has 120 million photo
receptors. It's like a 120 mega
megapixel camera. And each photo
receptor is just reporting how many
quanta of light, how many photons it
catches. So, I caught three, I caught
10, I caught 50. That's all you got, a
bunch of numbers. So, you have 120
million numbers. There are no colors.
There are no shapes. There are no
motions. There's just 120 million
colors. Numb numbers. Not not not even
colors. It's like if you look at the the
digital output from a video camera.
You'll just see a stream of numbers. If
you look at the stream of numbers,
you'll see the problem that vision has.
You can't tell from the stream of
numbers what's going on. You have to
create
three-dimensional objects and shapes and
colors and so forth from all those
numbers. And so that's the problem that
we have in vision. You have all these
photon counts, 120 million photon counts
on each eye. And from that you have to
then create objects. See that it's a boy
on a bicycle eating a hot dog. You know,
all of that is you. And that comes
that's not just theory. It becomes
really an important problem when you're
trying to build computer vision systems,
right? You're trying to build a
self-driving car, say with with passive
vision systems. Well, so the vision
systems are cameras, video cameras. Say
they're taking in video. Maybe they
have, you know, a few million pixels
that they come in that each each, you
know, maybe 70 times a second or
something like that. Well, those pixels
are just numbers. You got millions and
millions of numbers coming in every
second. There's nothing in there that
says that's a boy. That's a car. Don't
you know that's a stop sign. There's
nothing in there that says that. You
have to you have to have megabytes of
software that's really intelligent that
takes all those numbers and starts
computing with them to figure out
three-dimensional shapes to figure out
what the objects are to and to figure
out, oh, I'm about to hit a boy. I need
to hit the brakes and so forth. So, so
this is not just abstract. Self-driving
cars have to solve the problem of
starting with numbers that are in
unintelligent in some just a bunch of
numbers and giving you an intelligent
assay of what's happening in the world.
And so that's why a third of the brain,
literally a third of the cerebral
cortex, the higher part of of our brain
is involved just in visual perception.
When you add the other senses, it's it's
more like half the brain is involved in
sensory perception because the senses
are doing an incredibly complicated job.
But from my point of view, what they're
doing is they're building a VR world
and it takes a lot of processing power.
you need supercomputers, you know, what
would have been considered
supercomputers to do VR in real time.
And that's what we're doing. We open our
eyes and it looks like we're just seeing
a 3D world with objects and shapes and
colors. It seems so real and so just
we're seeing the truth. But because you
have billions of neurons, trillions of
synapses that are doing it all within
about a 100 milliseconds. And so you're
so fast at it that you just think you're
opening your eyes and seeing the truth.
You're seeing a VR world that you're
projecting out there in real time. Of
course, now I'm going to resend the
brain part, right? So the brain itself
is part of the space-time interface. So
the brain itself is just our VR symbol
for something deeper that's doing the
real work. And so the question will be
what deeper theory can we come up with,
right? It's going to be a theory outside
of space and and time. But this gets
back to what we have to do in science
all the time. We push our current
theories to the limits till they break.
And then we have to actually take a
creative leap. We can't necessarily just
use the language of our theories that we
just broke. We have to come up with a
deeper language. And that is a leap.
When Einstein
went past Newton, he took a deep leap.
And when quantum mechanics went past
Einstein and Newton, it took an even
deeper leap. And the language was
entirely different. But you can show for
example that if you start with Einstein
you get back Newton roughly as the speed
of light goes to infinity. The way
Newton talks about space and time and
matter you mass those terms actually
mean something different than what they
mean in Einstein. In Newton mass is mass
and you have the same mass period.
In Einstein your mass depends on your
velocity. Your length depends on your
velocity. Distances depend on your
velocity. None of that. Space and time
and mass don't behave that way at all in
Newton. We use the same words, but they
mean something very radically different
in Einstein. And in quantum mechanics,
it's it's even a deeper leap. But with
quantum mechanics, you get back Newton
as something called Planck's constant
goes to zero again roughly. I mean, this
is for first approximation. So what we
so the leap I need to make here now is
evolution by natural selection is
telling us that the language of space
and time and then objects in space and
time is the wrong language to describe
objective reality.
So the leap I need to make is to say is
there a deeper theoretical framework
that I can come up with
such that when I look at the dynamics of
that deeper framework and project it
back into our VR interface which I'm
which is space and time. So I've got
this VR headset of space and time.
That's what evolution has told us. This
thing is just a headset. You've got to
guess, right? I'm not telling you what's
outside the headset. I just told you all
I can tell you. It's a headset. Now,
it's up to you to take a stab at what's
behind the headset. So, that's what I'm
up to. So, I'm probably wrong, right? I
mean,
we weren't evolved. Evolution by itself
doesn't say anything that makes me think
I'm evolved to see the truth.
But it it at least is telling me that
whatever the truth is out there, you're
only seeing a headset. It's good enough
to tell us that, but it's not good
enough to tell us what's outside. that
takes a leap outside of the theory of
evolution. But the constraint on that
theory is we need to show how our deeper
theory could lead to me having a headset
and in that headset it looks like things
are evolving according to evolution but
natural selection. In other words, my
deeper theory has a strong constraint on
it. It better look like Einstein's
theory of spaceime. It better look like
quantum field theory. And it better look
like evolution by natural selection. the
three big pillars of modern science. If
I can't do that, then I know I'm wrong.
So,
even though all of those things exist in
the headset.
That's right.
Okay. This is [ __ ] fascinating. Uh,
so now
tall order.
Do you have that idea?
I have a proposal.
Let's hear it.
I'm probably wrong.
Can Before you give us your proposal, I
have I want to acknowledge you, dude.
The way that you talk like that, I love
so much. I can't remember if it was
Plank or Bore that said um anybody, this
is a terrible paraphrase. Anybody that
thinks that science advances because
objective truth is presented is
absolutely wrong. Science advances
because the old guard dies and the new
people grow up just believing it to be
self-evident.
I [ __ ] hate that so much. The fact
that people are not willing to be wrong
drives me crazy. Makes me want to choke
half the world out.
So the fact that you talk from the
perspective of, hey, look, I've got a
theory. It's probably wrong. Like, oh,
dude, I love that so much. I wish more
people were were as hardcore as you to
present you. You Oh, god. You call it
bold and precise. I I present an a
theory hypothesis that's both bold and
precise.
Right. Right.
Love that. And that you're willing to be
wrong. So anyway, I just had to take a
second. Every time I hear you say that,
I want to stand up and clap because so
few people are willing to own that
they're probably wrong, but they're not
afraid to make a bold and precise
prediction. So,
bravo. And and the the thing about that
is I just want to understand. And so if
I'm stuck on my ideas and won't let go
of them,
then if I'm wrong, I'm gonna not
understand. So it's really stupid to
think that you're you have to have
enough hope that your ideas have some
promise that you pursue them, but not be
dogmatic about them. That's that's a
it's a fine balance.
Very much so. All right. So now our bold
and precise claim is
so I'm proposing and I I'll say what I'm
proposing and then I'll say why I went
that direction. So I'm proposing that
reality is a vast social network of
interacting conscious agents. So that
consciousness is fundamental and think
of it like the Twitter verse, right?
There's
tens of millions of Twitter users,
billions of tweets, lots of stuff
trending. It's a Twitter users are
tweeting and following and so it's all a
big social interaction, right? So I'm
proposing and this is a mathematically
precise proposal that there are things
called conscious agents. So conscious
experiences like the taste of chocolate,
the smell of garlic are fundamental and
limited choices based on those
experiences. That's part of the whole
structure. So experiences that inform
choices. That's going to be the
fundamental idea
in a vast social network.
And the idea then about our headset is
is follows.
If you are a Twitter user and you want
to understand deeply what's going on in
the Twitter verse, well, you can't
engage with all 10 million users and the
billion tweets. It's just overwhelming.
You would die before you could even read
all the stuff.
So, what do you do? Well, whenever we
have big social media data, we have to
have visualization tools. Those tools
will necessarily ignore most of the
data. And the part that they don't
ignore, they're going to compress it
down. They're going to digest it and
compress it into some eye candy that we
can understand. Some objects in three
dimensions that have nice colors and
move in certain ways. And using that
visualization tool, I can maybe see
what's trending in New York, what's
happening in all, you know, so the big
scale of Europe, what's happening in
little scale in Irvine and so forth. So
I'll have a tool that lets me zoom in
and out and and it'll be ignoring most
of the stuff. And that's what I'm
claiming spaceime
is and physical objects. It's our
headset. It's a visualization tool that
certain conscious agents use to
interact with this vast social network
that would otherwise be completely
overwhelming. And so we've made the
rookie mistake.
The finish that the rookie mistake
rookie mistake of taking our headset for
the final reality.
Okay,
we have a tool and we thought it was the
thing we were visualizing.
Amazing. Very clear. Now, let's back up
and break these down part by part
because
conscious agent, I'm familiar enough
with your work that I kind of know what
you mean, but I don't think people
understand that like how small you take
that down cuz you're not talking that
there, oh, hey, this is all a bunch of
people, which is probably what somebody
hearing this for the first time thinks
that you think there are invisible
people that make up this social network.
Um, how far like, so turtles all the way
down, boys and girls, so we're talking
about consciousness all the way down,
right? How like are um nutrinos like are
they conscious?
Right. So no. So this is very different
from
is that because it's in the the headset?
That's right.
Nutrinos are particles inside spaceime.
Okay. Sorry.
Damn it.
Nothing inside is
I fell for rookie mistake. Uh okay. So
out outside
it. Uh so do you delineate between uh
advanced cognition and consciousness?
Not in principle. No.
Interesting.
So do you I'm really trying to get you
to use other words. So define agent.
Right.
So we're conscious. We're this is a
collection of conscious agents having a
social ex did you say social media
specifically? No you did not. Social
network.
Social network. They're like a vast
social network.
Got it. Okay. So, uh, give me what a
conscious agent is.
The simplest example, the most trivial
agent that the mathematics allows is an
agent that has only maybe two
experiences. Maybe like red and green.
Okay,
that's all it experiences.
Why does it have to be two?
Well, it could be even just none. You
could have an an agent that has none or
one. But I I tend to think about what I
call a one bid agent as sort of
fundamental. But I could have an agent
that has only one experience like
nothing or red
has an experience makes me feel like I'm
not I'm not interpreting what you say in
the same way that you mean it. So you
you said something and it went by so
fast.
Okay.
Uh which was the taste of garlic and
chocolate are fundamental if I remember
correctly.
Experiences. That's right.
Okay. So
literally the taste of garlic is a
conscious agent.
No, it's an experience.
Okay. So that thing it's not like, "Hey
Bob, you're the taste of garlic."
No. No. Okay.
So, sorry to use overly crass language,
but that sort of that was my initial
interpretation of what you said. Okay.
So, that makes sense. So, uh, I will
then push and say if that's the case, is
this not would not garlic and, um,
chocolate be tied to the envel?
Absolutely. So certain agents will,
but does that mean it's in the headset
then?
Um, well, those experiences, so the
headset is created out of your
experiences.
Uhhuh.
You So what an agent does is uses some
of its experiences as a format for a
headset.
But Okay. So what I'm trying to get to
and I think this is what you were
proposing is
hey we take the headset off and we see
like what do we see and I get that we
have the problem of perception and
that's ah the that's all back to the
envelope
see is a standin for obviously I don't
know how we would be interpreting this
world in the movie the matrix it is
green code right so when he explodes the
agents they explode into code
so code is their fundamental element Um,
what is your fundamental element?
Consciousness. I get that. But now I'm
trying to understand like how if it's
consciousness all the way down, what is
consciousness? Like is it a physical
substrate or is it not? And we have to
let go of the very notion of
physicality. Yeah, we're I'm letting go
of physicality in the sense I'm
completely letting go of space and time
and and particles, electrons, protons,
and neutrons. Those are only headset
entities. But is it fair to say that you
have no idea then exactly what
consciousness is? You just
Well, so so what we do so here's this
gets at the fundamental way we build
scientific theories and this is what we
talked about a little bit earlier which
is about every theory has miracles,
right? So every scientific theory
has certain assumptions that it makes
that we just have to grant you.
We just have to grant. No, there's no
theory of everything. We we hear about a
theory of everything. There is none.
So your miracle is consciousness.
That's right.
Okay. So there So I'm saying that there
are entities that I'll call conscious
agents. These agents themselves are not
conscious experiences. Why do they have
to beworked? Well, it it it turns out
that when agents interact, they form new
agents. So when and I didn't know this
when I wrote down the math. I was just
writing down what what could I possibly
mean by consciousness being fundamental.
And I wrote down the minimal structure I
could think of that could have some set
of experiences. So there's some set of
experiences that this creature, this
entity could have and a small set of
actions that it could take. And that was
all I wanted to write down. But then
when I had them interact with other
agents, right? Because the actions are
to affect the experiences of other
agents. It turned out the interactions
satisfy the definition of a conscious
agent.
So when agents interact, they form new
agents.
Let me ask a question that I think is
going to um explode this apart and help
us all understand what you mean.
Do I exist?
Um yes. In what way?
So you your conscious experiences exist.
So all I can see is your skin, hair, and
eyes. Yep. I see. But that's just my
interface symbol.
If you look at yourself in the mirror,
all you see in the mirror is skin, hair,
and eyes. But you know firsthand that
what you don't see in the mirror, your
hopes, your dreams, your aspirations,
your headache, all the rich world of
your conscious experiences, that is not
visible in the headset. So my conscious
experience is both in the headset and
outside of the headset. That's right.
That's right.
So it exists outside of space and time.
That's right. In fact,
this is where the [ __ ] starts to get
real weird.
It's pretty interesting because all I
can see of you are the experiences in my
own headset.
Mhm.
And so my headset is made up of my own
experiences that I've put in a
particular format of space and time. And
so so it's just a format. In a VR, for
example, you have a certain format in
which the VR is presented. that 3D VR
format has nothing to do with the shape
of the supercomputer. It's just the
format of of of that. And so this what I
see of Tom right here is just what my
headset allows me. But I believe and you
know firsthand that what I can't see in
my headset is this rich world of your
conscious experiences.
When I look at my cat,
my headset only shows me fur and
something really cute. And I believe
that behind that cute little icon in my
my interface, my headset, there is a
real consciousness, but I have less
insight into the consciousness than I
have into a human. With a mouse, even
less. With an ant, even less. And then
when I get to things that I call rocks
and, you know, protons and neutrons, I
have no insight at all.
No surprise. The whole point of a
headset is to simplify things. Does this
conscious entity need to eat?
Very interesting
because that gets to what? So, this is
now outside of space and time. It's not
evolution. It's not food and so forth.
So, the question is what are these
agents up to? What are they doing?
What's the why are they having any kind
of actions at all? And
the answer is I don't know yet. I've got
a ba mathematical definition of
conscious agent. were starting to play
with dynamical systems of them and as to
the question of why they would have any
dynamic why would consciousness do
something as opposed to nothing. What
what kind of answer could be deep enough
or at least what what kind of proposal
could be deep enough? And I I've only
had one idea ever that I've heard that
seems deep enough to at least be on the
table.
And that's consciousness all the way
down.
Well, no. I'm saying if we assume it's
consciousness all the way down, what
what are all those consciousnesses up
to? What is the social network doing and
why?
Okay, so the idea is the social network.
What's what's the one idea?
So So the one idea is the girdle is
incompleteness theorem,
right? So if consciousness is all there
is, conscious agents are all there is.
Yep.
Then mathematical structure is only
about consciousness and conscious agents
because that's all there is. And that
means girdle telling us that there's
endless exploration of mathematical
structure means there's endless
exploration of the possible kinds and
varieties of consciousness and conscious
agents. And what consciousness is up to
is is what I call the kid in the candy
store theory.
Girdle tells us there's an infinite
candy store of exploration of possible
conscious structures.
And the candy is all the variations of
conscious exper consciousness and
conscious experiences. That's right. And
that's I'm not saying it's right, but at
least it's deep enough that it could be,
you know, it's on the table.
Okay. So, here's the good news. I I am
the guy that's
uh dumb enough to like need everything
explained and hopefully that will be
useful to the audience. Uh, okay. So,
I've got this infinite candy store of
consciousness. The thing I cannot get
past is there's some there is utility in
creating the headset. Otherwise, why the
[ __ ] would it exist? So, if there is
utility in the creation of the headset,
first of all, I get it probably begs
questions that you you don't know and
it's beyond the the scope and you've
already asked me to just accept that the
miracle is consciousness and you're not
going to tell me anything beyond that
because nobody has a theory of
everything. Respect. Get that. Uh but I
want to poke in the spirit of fun partly
and then just to see like where the sort
of edge cases are. All right. So we
don't know if conscious entities need to
eat. And the reason that I asked that
question was because all of this starts
with you looking at fitness and so if
there is fitness into taking multiple
bites of the apple therefore logic is
going to be born out of that so that I
know to keep eating the apple. Um but
then it begs a question of well what am
I underneath the visor underneath the
headset just just to keep our
nomenclature consistent. What am I
beneath the headset? And why does it
matter that I have a representation that
that is based in the idea of fitness of
eating an apple? Right? So that's where
I'm like what what is that a
representation of? Why does this
representation need to exist? like I'm
sure you've daydreamed about this even
those are the fun questions right that's
that's what we're really interested is
answering those questions and so we're
going at it in two different directions
one is you know sitting back in our
armchairs and trying to think like
girdle's incompleteness theorem and so
forth and saying could this be and
can I repeat what girdle's
incompleteness theorem is it's the one
part of this I'm nodding and smiling but
I'm like the [ __ ] is he talking about uh
so it goes something like this and you
will tell me where I'm wrong. So, uh,
in fact, God, can I even articulate what
I think? I'm I'm going to fumble through
this.
Okay,
I I don't even I feel like I understand
a middle piece of it. I don't know where
it begins or where it ends, but that we
have some sort of mathematical equation
that says you're never going to find the
limit to which you can uh explore one
thing. But I don't get why that thing is
Girdle's incompleteness serum has
nothing to do specifically with
consciousness, right?
somebody leveraging that to explain why
is it then that consciousness is
the only thing that you've seen put on
the table that
uh ties into that.
Oh well so okay the reason I went after
consciousness was I' I've been trying to
solve what's called the hard problem of
consciousness. How is consciousness
related to brain activity?
Yes.
And so people have been trying to show
how consciousness can be booted up from
brain activity and we've utterly failed
to do that. or how the illusion of
consciousness could be booted up from
brain activity. And there are no
mathemat absolutely no mathematically
precise theories after decades of effort
that could explain even one specific
conscious experience or one specific
illusion of conscious experience like
the taste of vanilla or why we why this
kind of brain activity must be the
illusion of the taste of vanilla. Why it
could not be the illusion of the taste
of chocolate. There's nothing on the
table. I mean there's no science that
can predict even one specific conscious
experience or one specific illusion of
conscious experience. And so the reason
I went after consciousness being
fundamental the reason I went after that
that's different from girdles
incompleteness there was that I didn't
want to be a dualist right so when as
scientists we try to create a theory
based on as few assumptions as possible
and we only want one kind of assumption
we don't want to have like I want this
physical stuff and I want this
consciousness stuff you you you have to
choose pick either physical stuff that's
unconscious or pick consciousness stuff
that's not physical, but don't don't do
both. If you do both, that's dualism and
and that's not is not as clean. Maybe
we'll have to, but we don't want to go
there. So So maybe dualism will end up
being where we have to go. But I'm not
going to start there. I'm going to start
with physicalism doesn't seem to be
working out. It seems to be principled.
Physicalism is that you can stack enough
neurons together that they suddenly
become conscious.
That's right. And space and time are
fundamental. And that's where this what
evolution is basically saying.
Physicalism is false. That's why I went
after evolution. Natural selection says
the language of space and time is not
the language of objective reality. That
means physicalism as we currently
conceive of it is false.
We can't boot up consciousness from
neural activity because neural activity
is just
a data structure in your headset.
Neurons do not exist when they're not
perceived. They couldn't possibly create
consciousness. They're not even there.
Consciousness, I'm proposing, creates
neurons when we look inside skulls. But
neurons could not possibly create
consciousness. And I was forced to that
by looking at natural selection. So I'm
proposing that if I believe that I
really do have headaches
and I really do taste chocolate and I
really do have conscious experience. And
again, I could be wrong about that. It
could be an illusion.
And just to be clear, in fact, we never
finish this because I'm still I'm just
paralyzed by Girdle's incompleteness
theorem. But we're going to move on from
that to not browbeat the poor audience.
Um, so going back to uh this network of
conscious agents you're saying and and
this gets into the part about the brain
I wanted to talk about. So split brain
patients can have an experience of being
literally two separate people. So you go
in, you cut the corpus colossum. So
everybody to your point for a minute I'm
going to assume that the brain actually
exists. Sure. Uh so you go in, you cut
the corpus colossum which allows for
communication between the two
hemispheres of the brain and suddenly
you realize that two personalities will
emerge within the same head. Um do they
they don't both have internal dialogue
do they?
They have different likes and dislikes.
But do they have an internal because one
side handles language. So they both do
have
they both have language. Actually the
right hemisphere is very adept at
language.
Fascinating. So they both have language.
One could be an atheist, the other
devout, which is so crazy. And that's a
real example right from the literature.
There was
my friend VS Ramondrin has a video
online. You can even Google that and
say, you know, split brain patient
Ramicondrin. This patient has, you know,
believe atheist and a believer. And one,
you know, I think in his case, the right
hemisphere didn't believe and the left
hemisphere was a believer.
It's crazy. I've never seen the video,
but that's
Oh, it's great. One case, the right
hemisphere wanted to be a race car
driver. The left hemisphere wants to be
a draftsman. They have just completely
different personalities. They can play
20 questions with each other. You can
give a word to the right hemisphere and
the left hemisphere will will sometimes
fail in 20 questions. They can't figure
out what's in the right hemisphere's
head.
Whoa.
So there so there are separate contents
of consciousness. So separate that you
can lose at the game of 20 questions
with your other hemisphere.
That's insane. Right. So you're that
would be a very sort of simplistic
example of two different conscious
agents that have come together.
That's right. That's and that's the
idea. So you are one conscious agent.
I Well, the way that I perceive myself
is as one conscious agent, but aren't
secretly I just a whole bunch of
conscious agents.
Well, that's that's that's the thing.
You are it's both are true. So you are
one conscious agent, but you're also
two. And you're also probably a huge
lattice of interacting conscious agents.
You know the microbiome. Yeah,
a little bit. Yeah.
Okay. So in your gut, you have like
trillions of bacteria. It's crazy. You
have more foreign cells in your body
than you have your own cells.
Uh would they be conscious agents?
Nothing in spacetime, strictly speaking,
is a conscious agent.
Do they not represent something? That's
another thing I've resisted asking you
this question because I think I know how
you will answer it.
The very like no one knew the microbiome
existed,
right?
So if this is all in my headset, how is
something so detailed that once somebody
looks it's like oh my god it's
proliferating? It there's all this
stuff. How can we discover something new
if it's literally just made up? Like the
first time that somebody cracked open a
skull, why did everyone look at it and
see the same brain? The first time we
looked into the microbiome, why do we
all see the same thing? Shouldn't a
novel thing that evolution did not
prepare us to see and understand, like
how do we all see the same thing? Now, I
think I actually know how you're going
to answer it, right? Because once you
have a good visualization tool, it
depends on where you take it. So, I
could take my visualization tool for the
Twitterverse, right? And I can zoom in
on Irvine to look very, very close. And
I can zoom back and look at the whole
United States and see what that looks
like. So, a good visualization tool for
a social network lets you zoom in and
out.
Doesn't that assume that everything Oh,
god. Let's see if I can articulate this.
Doesn't that assume that tweets are a
substrate that is universal? So, that uh
the the first image that came to mind
was
why the hell the first time somebody got
their head bashed open did everyone see
brain the same way? And I thought, okay,
well, he's going to say that the reason
they did is because you have a faximile
in the headset for um
tissue that's made from atoms. And so
like that that visual structure, it
doesn't care like what it is. It's photo
receptors taking in. I see this much uh
light on this receptor, this on that,
all that. So it's like, hey, I have a
system for dealing with visuals and
therefore when photons bounce off of
this thing, it's going to construct
something. And I understand things about
3D and mushiness and texture and all
that. And so my brain is programmed for
that. So no matter what you put in front
of it, it's it's going to see that. I
assume that is correct
roughly. But this is the it's more like
if you are trying to look at the Twitter
verse and you have a you if you design a
really good tool and it let you zoom in
what's happening just in your block
versus in your city versus in your
county versus in your state versus your
country versus the whole world versus
Europe and so forth. If the tool is
really good, it's going to let you you
will see different kinds of structures
as you move in. Maybe it's very very all
the same in my block where you know we
all have similar ideas and we do the
same thing or in my my county but it'll
be very different and so so the reason
we see right when you look inside of a
brain you are not just and you see all
these neurons and so forth you're not
just one agent you're two you're a whole
lattice of conscious agents what we're
doing is using our visualization tool to
look at the whole list of conscious
agents that are together forming you. So
we're using that visualization tool to
look in finer and finer detail at agents
that perhaps are having smaller and
smaller sets of conscious conscious
experience,
but we don't really know that it's a
onetoone relationship between a cell or
whatever and what else?
Yeah, it's going to be many to one.
That's right. So for example, I mean I
see someone I call Tom in front of me,
but that's I see one Tom, but I my
theory is proposing that that there are
countless conscious agents that I'm
interacting with. There's one highest
level conscious agent, but immediately
below it there are two that I associate
with what I call the left and right
hemisphere. And then below each of
those, there's countless more. And
there's more than one personality in
Tom. The right and left hemisphere
agents probably have very very different
personalities. It seems to be a general
trend. They're they're very very
different. And who knows what goes among
all those agents all the way down. So my
visualization tool, of course, all I see
right now is skin, hair, and eyes. That
is pretty simple compared to what I'm
claiming you are.
Really good-looking skin,
very good. And it's a really complicated
and intelligent the network of conscious
agents, but I see just very very little.
But when I look in when someone looks
inside and sees a brain, the 86 billion
neurons they're seeing there is my
visualization tool telling me there is a
lot of conscious agents in a really
complex social network going on here.
That's what I'm seeing as 86 billion
neurons. Then when we get down to
chemistry which is you know explode
that's even more more complicated now
you know saying well my interface is
starting to give up because you're not
seeing much about consciousness with
with neurons you might be getting some
notion of networking and exchanging
information maybe at the you know
biochemistry you're not seeing that
quite as well when you get down to you
know quarks and gluons you may be giving
up but there's tons and tons of quirks
and gluons and that's my interface
telling me look I can I'm showing
showing you a lot about Tom. I'm not
showing you too much about his two
hemispheres, you know, the two agents.
I'm showing you very very little, but
you know, I'll show it to you what you
call, you know, 86 billion neurons. And
then and eventually my interface is
going to have to just give up because I
mean, the whole point of the interface
is the network of conscious agents is
too complicated for you to gro.
You can't gro it.
So we I'll give it to you one agent at a
time. Here's Tom.
Not even Tom's left and right
hemisphere, just Tom. And then all the
agents if you want to you know you can
get out you know if Tom will let you he
we can go in there and look in his brain
and we can
Tom will not let you.
Yeah I don't think he will. That's
right. I wouldn't. That's right.
So that's why the tool
is showing us more complexity all the
way down. It's it's a vast social
network and each agent isn't just a
standalone. We're we're a combination of
many many other agents. That's going to
be one part of the theory that's really
interesting is mathematically precisely
looking at all the ways that agents can
combine.
But agents do combine. Uh we'll be going
to some new mathematics I think
something called um infinite categories
and topo where
infinite categories what
infinite category theory and it's called
infinity categories and also to theory.
Some
tollo to oi. Right.
What the hell is that? It's some fairly
abstract mathematics that allows you to
economically start to describe countless
other ways of interaction. And so what I
want to I mean I have a few ways that my
current mathematical model which just
uses things called marovian kernels and
measurable spaces
obviously
plotting for what I mean most most
mathematicians would would say it's it's
fairly plotting but but it's it's it's
it's it works and it's it's real math.
we can start with that and then go to
these, you know, category theories and
so forth to actually get the full
richness of all the kinds of ways that
they could could connect and interact.
And so I I want to go after that. Um,
but the the one question you asked
earlier was
why does it look like we have to eat and
do the conscious agents themselves have
to eat and what are they up to and how
can we know? And here's here's my answer
to that. First, I don't know the answer.
So, so here's how I'm going to go after
it. First, I I am thinking like girdle
is incomplete and certain. We can come
back to it. That might be a deep
motivation for the whole dynamics. But
suppose I'm that's wrong and I realize
it's wrong and I realize I'm just not
smart enough and my team is just not
smart enough to figure out what
conscious agents are up to and why
they're doing it and so forth. What the
Here's what we're going to have to do.
We're going to have to propose a
mathematical projection from the
dynamics of conscious agents into space
and time
and propose how that dynamics gets
represented in terms of for example
quarks and gluons interacting. Right now
I'm studying scattering amplitudes at
the large collider. I want to show how I
can predict scattering amplitudes when
two gluons hit four gluons go spraying
out from the dynamics of consciousness.
The the reason for doing that is that if
I'm not well a couple reasons, but one
is if I'm not smart enough to figure out
what consciousness is about, what I'll
then do is say, "Here's a mapping in
that gives me back spaceime." Now, now
that I know what what's happening in
spaceime, I'm going to pull it backwards
and say, "What does that tell me about
the dynamics of consciousness?" And I'll
go, "Oh, I never thought about that."
So, so conscious agents have to be
interacting this way for it to look like
this in my headset. So if I'm too stupid
to figure it out and it's very very
likely I will have to take my theory of
conscious agents projected into my
headset and say what would it look like?
Oh I'm getting the wrong answer. Okay so
I need to change my dynamics of
consciousness this way so it really
looks like the scattering amplitudes of
quarks and gluons in the large hadron
glider. And when I get that match then I
go okay this is at least one dynamics of
consciousness that gives me the right
answer in my headset. Now, what is it
telling me about what consciousness is
up to? So if I'm not if we're not smart
enough to do it from first principles
and I want to do it from first
principles but if I can't I'll try to
reverse engineer relativity theory
evolution by natural selection quantum
mechanics reverse engineer all of those
pull them back to the realm of conscious
agents look what they're saying about
the social dynamics and then try to get
a clue about what that's about and
answer questions like am I forced to
think that they need to eat or is there
some deeper principle is is it for
example that I can only send an
experience to another agent if an agent
sends an experience to me. Is there is
there that kind of dynamics that and and
is it going to be like um the small
world networks we see for example in the
internet where you get big hubs
like Google and Apple which get lots of
hits and then tiny little guys like
Kaufman who gets almost no hits and then
a few in between but but but you get
this and and there's in some sense the
number of social connections you have is
some sense your notion of fitness is the
creators of Google are billionaires.
Hoffman's not a billionaire. There's a
correlation between the number of hits
that Google had gets, the number of hits
Hoffman gets and the difference in
richness. Now, is that a deep
property of these conscious agent
networks or not? See, these are the and
this is what is fun about science. I
don't know the answer. Evolution by
natural selection has told me it's not
in space and time. It's saying it's not.
This is all a headset. You're gonna have
to think as a scientist out of the
headset, but I I can't tell you what's
outside the headset.
So, we have to be very imaginative
and we have to choose what we're going
to go after because of course we're
probably going to be wrong. So, you need
to choose what you think is going to be
interesting. I'm going after
consciousness because I'm trying to
solve this hard problem of how is
consciousness related to our brain
activity.
And I'm the mathematics is forcing me to
see this vast social network. If I bring
it to consciousness, there's this whole
network, but I don't know what it's
about.
All the science that we've ever done,
all the science that's ever been done so
far has only been in our headset.
Quantum field theory assumes spacetime.
The fields are defined over spacetime.
Relativity is spacetime
and evolution by natural selection has
been about what evolves in spaceime. All
of our science, which is very, very good
science, has been in our headset. We've
never really stepped out of our headset,
but science has the tools. There is some
initial work. There's a guy named Nema
Arani Hamemed at Princeton, Institute
for Advanced Study, Princeton, who I
think is taking some really important
steps beyond the headset because he's
he's already realized spaceime is
doomed. And so he is already being very
adventurous and stepping outside of
spaceime looking for mathematical
structures in which space and time and
quantum mechanics do not appear. All
right, I'm there's something I find
super interesting in this whole concept
of spacetime is doomed.
Right?
If we abandon spaceime,
it begs a question about causality. So,
right now, I cannot believe this is
true, but this is true. On Monday, I
have two memorials to go to. Young
people, crazy. I'm literally
heartbroken. And the fact that they both
have memorials on the same day is too
weird to even contemplate.
So, that got me thinking about the very
nature of death. But if time doesn't
exist,
is there like so causality requires
time? It requires you to be moving
through time for something to happen
first that then creates another knock on
effect, another knock on effect, so on
and so forth. But if time doesn't exist,
and I mean admittedly, we don't know
what it's being replaced with, but
theoretically we know that it isn't
going to be like we perceive it now. So
if time doesn't exist then
like where is that notion of entering
the the time stream of breaking out of
seeing this as a a linear equation and
obviously tying it to death being
I would love to think that there's some
way to once again interact with these
people if I can just take the headset
off.
Um do you have any intuitions or is
there any math around like if it if it
isn't for let me just ask one question.
Is cause and effect real outside of the
headset? I think inside the head inside
the headset, no. So, we have a useful
fiction
inside the headset. You don't think
there's cause and effect?
Right. Well, we have the appearance of
cause and effect, right? So, if I hit
the the Q ball
Mhm.
and it hits the eightball into the
corner pocket, it looks like the Q ball
caused the eightball to the corner
pocket.
Yes.
And that all works, but it's a fiction
of causality. In the old video game of
Pong, you have these little paddles and
this ball and you hit and it looks like
the paddle is causing the ball and and
the fiction of causality is good enough
that you can play a game. You can
actually figure out how to put maybe a
little spin on it and so forth. Or in
you know more more advanced games like
you know virtual reality or games or you
know like Grand Theft Auto, you have a
nice fiction of causality. I turn the
wheel to the left, my car goes to the
left, turn the wheel to the right, but
it's all a fiction. The wheel has no
causal powers. The gas pedal has no
causal powers, but it's a useful
fiction. Evolution gave us a
usefulction. I don't know that that's
true. So, think about
a system was created. Let's let's take
the analogy of Grand Theft Auto. So, a
system is created such that it awaits
input from your your control pad,
whatever that control pad may be. Now,
admittedly, my control pad, I press it,
an electrical signal is sent to
something that you turns on or off or
opens or closes or whatever, and then a
whole cascade of things happens. But it
it is I mean you could even trace back
the causality to I ate something that
gave me the ability to create ATP which
gave me the ability to generate
electricity which gave me the ability to
fire a muscle which gave me the ability
to press a button which triggered this
electrical chain reaction that caused
something to happen on the screen. But
it it there is a chain of causality in
the headset. There is a chain of
causality in the headset. like even even
though it is the perception is that you
know I'm uh turning and the act of
actually just moving that thing makes it
turn. So I get that that isn't happening
but it does create this chain reaction
that can be in the headset understood.
Right. And so there I disagree. So if
you look for example in a a VR version
of it where you see your virtual hands
holding a virtual steering wheel
Yep.
There's no feedback from the screen to
the computer.
Uh there's no feedback from the screen
to the computer, but there is feedback
like if you think about VR. So,
but so there is a there is a real cause
and effect, but it's not what we see in
the headset. It in the headset it looks
like the wheel that I'm seeing in the
headset in the VR headset is what's
causing the car.
Yes. But I am typically in VR I would be
mapping my real hands either by holding
a controller or by having cameras.
Absolutely. So, I'm mapping my hands. My
hands do a movement and this program is
programmed to wait for the input from
the movement of my hands to then trigger
that sequence that I just listed before.
So, there is a cause and effect.
Yep.
But but notice it's not from the things
that you're literally seeing in the
headset.
I agree with that. But I'm saying but
there there still is a chain of cause
and effect. Because when you say that it
is an illusion of the eightball hitting
the thing, it's like that doesn't seem
true within the construct of the
headset. I'm saying
right. Right. Well, so so maybe another
example might help on this. So when you
like drag your icon, you you've written
a email and it's the icon is blue in the
middle of your screen, you drag it to
the trash can.
Yep.
You you are using say a mouse or a
touchpad and and in so in this analogy
that would be like the real cause and
effect. But if you said, "But it's
really the motion of the icon on the
screen to the trash can that's causing
the file to be deleted." That's just an
illusion of causality, right? The
there's no feedback from the pixels of
the screen into the computer. And that's
what I mean. Everything inside space and
time is like seeing the icons on your
desktop. Things move around on your
desktop, but it's due to something like
your, you know, joystick or something
else. But the headset, since the headset
is all of space and time, what that
means is that I'm saying that everything
inside spaceime is part of that
fictional causality.
I now get what you're saying. So in the
in the headset, that perception is
illusion. And I was dragging us sort of
back and forth between
But it's good. I mean, that's very very
good.
Now take me outside. Does cause and
effect exist if time doesn't exist?
So So two answers and I'll get to the
death question. Um
I think in the realm of conscious agents
there is a notion of cause and effect
but it it boils down to a notion of free
will. That seems to be one of the key
notions of cause in in in this in
quantum mechanics. By the way I should
point out that
they when you do quantum computations
normally in normal computations there's
a causal order. If you do this multiply
and then an add that order is important.
If you do the add and then the multiply
you get a different answer right the
order the causal order but it turns out
in quantum mechanics you can do a you
can get rid of causal order when you do
your computation you can have a
superposition of you know multiply
followed by add with add followed by
multiply both orders so you have a
superposition of the causal order and
it's a theorem that in general you'll be
faster you'll be more efficient if you
let go of causality in spacetime and it
turns out that's actually been done.
When you build these things and let go
of causality and spacetime, you tap you
tap into a greater efficiency. So when
you let go of spacetime, you're also
letting go when what when spaceime is
doomed means also that everything that
we believed to be causality in spacetime
was just a very useful fiction in our
headset. Just like the fictions in
various VR games that we play, it's a
useful fiction. Lets us play the game.
um there is an underlying causal order
and we're arbitrarily ignorant about
what that causal order is. Now, about
death,
I don't know. But here's an here's an
interesting idea. Suppose you go to a VR
arcade with some friends to play virtual
volleyball and you put on your headset
and bodysuit and you're on a, you know,
a
like a beach volleyball scene, palm
trees and sand and the net and you're
playing VR volleyball for a while and
then one of your friends, you know,
says, "I'm thirsty. I need a drink." He
takes off his headset and body to go to
go get a drink.
His avatar sits lifeless on the sand. It
collapses on the sand. It looks within
the VR headset, within the game as
though he's dead, but he's his
consciousness has not ceased. He's
merely stepped out of that interface.
All right. Now, we're in what I'll call
the Phineas Gauge problem.
Okay.
So, once So, Phineas gauge, for those
that don't know, it was a railroad
worker. One of the most famous examples
in neuroscience. He's hitting a tamping
rod. It's like a three-foot rod, thicker
than your thumb, and it shoots up
through his jaw and out the top of his
head, taking, if I remember right, a
teacup's worth of brain matter, which
seems impossible, but never loses
consciousness, but they say he's forever
different, right?
Different
and he used to be like super sweet and
he was one of the best workers and then
he becomes this belligerent [ __ ] and
he can't hold a job, right? So I will
say that's that is the so using this
notion of the headset or the envel. It's
like once you alter the way that his
brain works and I'm fully willing to
accept that this is a problem only
inside of the headset. But once you
alter that function within the headset
he's fundamentally different. Now, if
the headset is is what I'll call the
unvelt, it's our interpretation of the
stimuli. Once he goes takes the headset
off, the inter now we're into
transistors, diodes, electrical,
gravity, or oh god, I may have tripped
up with gravity, but everything shy of
that one.
Um, now you have he the experience of
that person would be fundamentally
different. I would be fundamentally
different because I'm no longer
experiencing things through the lens of
my brain essentially, right? So while I
it would seem to me that there's no way
around the fact that in the game homie
is dead and so or lifeless to use your
example. So
since that's the only thing the I want
to relate to that person through my
headset, right?
That's what I'm used to. That's where
the emotion lies. And then now once I
step outside even emotion would be
called into question almost certainly in
fact definitively in your explanation
the way that I process emotions it would
just be unrecognizable outside because I
no longer have the brain I no longer
have all of those things. Interesting.
You said quite possibly.
Right.
So there is a possibility to you that
outside the headset it's
close to what we experience inside.
Right. See so for me um it's going to be
a matter of following the math on this.
So the theory of conscious agents itself
doesn't require an agent to have a self.
So
meaning it can shatter into a bunch of
little pieces.
Well, that that a self is something that
a network of conscious agents has to
construct.
Mhm.
So are we the Borg?
It would be
like an interface representation.
Okay. So my what I call myself
is on this kind of view no less a
construction than space and time and so
it's not clear to me how much of that
construction will survive death will
meaning taking off the headset
taking off two possibly very different
things. Well, it looks for for people
who still had the headset on,
it would be indicated.
It looks to them like death, but for the
person who's actually having the headset
taken off, maybe it looks like what some
people describe with psychedelic
experiences or near-death experiences
and and so forth or or you know, like
really extreme five DMT MODMT
experiences and so forth. Like the these
are things I want to explore. So at this
point I I'll have to say you know
not only you know I'm likely to be wrong
I'm still trying to figure out what the
ideas are that I would want to put on
the table that are wrong. So but the
theory of consciousness that I have
right now doesn't require conscious
agents to have a self to have memories
to have the ability to learn or anything
like that. So networks of conscious
agents construct selves, they construct
memories, they construct emo, you know,
patterns of logics of emotions and so
forth. It's going to be very interesting
to to ask the question about how much
when you know I my parents died
recently, right? So, you know, it's very
very it's very very difficult, right?
You
and
so we all would like to think that there
is
some way to have contact with a person
anyway. If you're a physicist, of
course, that's out of the question,
right? If your consciousness is nothing
above and beyond brain activity, then
when brain activity ceases, there's no
consciousness. End of story. It's very,
very clear. In the theory of conscious
agents, it's quite possible that that
yourself dissolves. But the conscious
agents, it seems to me, will still be
conscious agents. they've just maybe
dissolved a particular
data structure that they created a
space-time data structure with a
particular personality and memories and
so forth. So basically you're saying uh
if I can just put different words around
that to make sure that I understand
the the conscious agents coming together
to form a person are a data structure
that well they form the person as a data
structure. So everything that I believe
about me is very different than what I
believed when I was five.
Mhm. and and I didn't believe anything
when I was maybe one. And so I've over
the years, the decades of my life, I've
put together this story of I'm myself.
This is sort of an idea that some of my
physicalist colleagues have said like
Dan Dennett, a self is a story that we
weave. It's a narrative that that we
give. Maybe there's something to that.
Whereas where I wouldn't go with Dan on,
you know, and he's a wonderful guy. He's
he's brilliant, but you know, we we are
it's okay to disagree, right? He would
say that, you know, there's nothing but
the brain activity and the narratives
that it creates. And I'm saying, well, I
like the idea of the narratives, but but
the consciousness is the fundamental
reality. I have to let go of physicalism
because of evolution of natural
selection. And so maybe consciousnesses
won't keep the narratives. Maybe they
will. I have some of my colleagues who
are working with me who think um you
know that we will keep the narratives
that we will keep that sense of self.
But my colleague who thinks that doesn't
have any mathematics to support it. And
so for for me on the one hand of course
I'm really open to all the different
ideas but as a scientist if I can't put
it precisely in math I don't know what
I'm talking about yet. I mean you and
that's what you find is unless you can
make it absolutely pre precise most of
the time we think we know what we're
talking about and when we make it
precise we realize oh okay no I was
maybe in the neighborhood but I didn't
really know what I was talking about.
That's what mathematics really comes
back and teaches you. And that's the
thing about a really good scientific
theory. Once you write it down, you
become a student of your theory. Like so
Einstein when he wrote down the equation
of general relativity.
He had the big idea falling in an
elevator you would feel weightless. Big
big idea. If it was free falling
elevator and he took him eight years to
take that idea and what he called the
equivalence principle and turned it into
the equation of general relativity. He
wrote it down. Took him eight years.
hard hard work to take your intuition
and go oh no my intuition what is what
do I really mean by that took him eight
years and he was Einstein right right
for the rest of us I mean if Einstein's
ideas aren't that quite precise it take
him eight years to get it so precise
that he knows exactly what he meant
that's what that's what I'm talking
about so he writes down the equation a
year later
a guy named Schwarzild who's in the
front lines in World War I
is solving Einstein's equation on the
front lines and he solves equations and
he discovers black holes and he writes
back to Einstein and says your theory
says there are black holes. Einstein
didn't know that. He didn't believe it.
He spent decades dis disbelieving it.
His equations were right. Einstein was
wrong. The equations become smarter than
the genius who wrote them down. And
that's another reason why we do these
mathematical models of science. We take
our intuitions. It might take us a
decade to take our intuitions and
actually figure out what we really were
thinking and get them so precise that we
say, "Oh, that's the only logically
consistent way of stating what I thought
I was trying to think." And once you've
put it down there, then all of a sudden
you become a student. That thing that
you've written down is going to teach
you. And that's what I have with this
theory of conscious agents. When I wrote
it down, I had no idea about a number of
things. I didn't know that agents could
combine it. It was a friend of mine who
pointed that out to me. Said agents can
combine. And also that that a weird
thing that it predicts is that our free
choices are not part of our conscious
experiences. You can't directly
experience your own free choices. You
can experience that you chose, but you
can never actually experience yourself
choosing.
Why?
Well, it's it's really quite
interesting. you can experience like if
if if I go, you know, here's there's
chocolate and vanilla. I'm going to
choose between chocolate and vanilla.
Well, I I just chose chocolate. But how
did I do that? Well, I had some
deliberation process, but when I finally
it I can all I can do is see myself
reaching for the chocolate or the
vanilla. I can see my cogitation
processes, but but the
But isn't that all in the headset?
All that I'm seeing is in the headset.
That's right. So I'm seeing. So by the
way, I only know my actions
through my headset. I actually don't
know what I'm doing. When I reach out
and grab something,
I don't know what I'm doing in objective
reality in the realm of conscious
agents. How does all of this play out in
your real life? And I' I've heard you
talk about that there are moments where
you have I'm sort of putting words in
your mouth, but almost a meditative
experience where you transcend the
notion of self. Um,
how so?
How do you stay so enthusiastic about
this for so long when it seems like I
mean really really at a deep [ __ ]
level, man. As a human experience, this
just all feels so real.
Right. Right.
Well,
I wouldn't say that I transcend
the self, but I what I
do get it once in a while is a glimpse
that oh,
this is just a headset. I I actually
feel it that I'm just rendering this.
Most of us feel like space just exists.
I'm stuck inside space. There's this big
stage. I'm on the stage. It's very
different. I think, by the way, the next
generation will probably get this much
easier.
Those who have just been raised, they
spending a lot of time in VRs that are
as compelling and as immersive as
everyday life. It's going to be just
sort of obvious that you take your
headset off and go, it's a no-brainer to
think, well, this is just a headset,
too. And to just sort of be there. So, I
think that it'll it'll be
for the next generation. The fact that
I'm having a hard time about it thinking
about it this way and imagining it
experientially will just be sort of an
artifact of the technology I grew up
with. If I grew up with VRs that were
really good as opposed to the stuff that
we grew up with, which is not that good,
then it would just be sort of obvious.
You do it when you're young enough, it
just is obvious that that Yeah, I'm just
seeing a VR headset, too. By the way,
here's one way to think about it. If you
close your eyes, you just see sort of
gray, right? Modeled gray in front of
you. So it looks it doesn't look like
nothing. It looks like modeled gray. But
what is it like backwards back through
your head
when you close your eyes? Well, it's not
modeled gray. It's nothing. And it's
really the first time you really if you
close your eyes and experience that.
Yeah. What is it like in front of me?
Yeah, it's just gray. Sort of modeled
gray. What is it like behind me?
Absolutely nothing. That's the headset.
You only have a headset of spaceime in
front. There is no headset behind. Now
you have a not a visual headset. Now you
have this set. I can put my hands back
there and do stuff. So So I have this
but it's all a creation. It's all so I
do get glimpses of that
once in a while but there were no put on
the natural selection language. Right.
So I have to pick the language of the
science that I want to use you know
because I don't have a better language
in some sense for for discussing this
evolution. Um there were no selection
pressures for us to see the truth.
And so there were no selection pressures
for us to not take spaceime as the
truth. And so we do. Pay tells us you
know when
we begin to take objects as real as we
you know these aren't just like little
data structures that you create that
they really exist all the time. He
called it object permanence. And PJ said
that you know when a kid is about 17
months 16 or 17 months of age they don't
have object permanence permanence. You
take a little baby doll put in front of
a child they play with it. You put put
it behind the pillow. If they're 16
months old, they
just doesn't exist.
It doesn't exist, KJ said. And then but
at 18 months now, they go and crawl
around and try to get the object, the
baby doll out of the behind the pillow.
Later experiments showed maybe down to
three or four months. But the point is,
these experiments show that we're
programmed, now I'm using the evolution
language, we're programmed by natural
selection to buy into the illusion that
objects exist even when they're not
perceived. Object permanence when we're
three or four months old. We're not
rational. It's being done to us without
our permission. And so by the time we
come to the age of reason, it's it's the
water that we we don't know that we're
wet. It's the water we've been swimming
in all of our life. We just have been
programmed to take this as the reality.
I took it for the reality. It was only
because I couldn't solve certain
problems like the problem of
consciousness. And it was only because
when I looked at evolution, it began to
tell me spaceime is not the reality. It
cannot be. It's the wrong language. It
must be only like a headset that I was
going holy. I mean, I still remember the
first time I realized this must be just
a headset. I had to sit down. It was
such I mean, I was a grown adult. I was
like I I was around 30 years old or
something like that.
The first time I realized this, it was
such a shock. I had to sit down.
Everything that I believed all of a
sudden disappeared. But of course, the
next moment I was again visually
believing I'm in reality. I'm seeing the
truth. So the programming is there, but
ever since that that moment when the ma
it was the math that that did it to me.
So
who do you think created math?
Very interesting question.
my own
thinking in terms of this idea that
consciousness is fundamental and
conscious agents are fundamental
when we actually study consciousness and
there's been a scientific study of
consciousness since 1860 it's called the
field of psychopysics there's a guy
named Gustaf Fner who started the whole
field and a lot of my research has been
in psychophysics where we literally get
mathematical models of conscious
experiences and we test people very very
carefully in the
We find that mathematical exper that
conscious experiences are mathematically
structured. My experience of this water
bottle
the mathematics is unbelievable. There
are you can write down differential
geometry you reflectance functions. I
mean the mathematics is incredible.
It's true of all of our conscious
experience of everywhere we look.
Conscious experience we it seems so
squishy goes through your fingers. How
can you? There's mathematics. So the way
I think about math and experience is
that mathematics is like the bones of
the living conscious experience. They're
not they can't be divorced from each
other. There's more to experience than
just math, but there's not less than
math. There's math and more. And so that
mathematics and conscious experiences
have a deep intimate relationship that
I'm still trying to understand. But but
but the empirical
evidence is quite strong. I mean all the
psychophysics that people have done, we
just find mathematical structure
everywhere. And that's why I came back
to this girdle's incompleteness theorem
where where where that theorem is just
saying
no matter how many mathematical
structures you discover, you haven't
started. There will be endless more
structure come from.
Girdle
figured out how to do this. So it came
from
and can I let me maybe now I'm
understanding it. So
we once thought the atom was the
smallest structure and then we
discovered there's something below that
and something below that and something
below that. Are you saying outside of
the headset there's just no end to the
something below that?
That's right.
Interesting. That's right. that and and
this is what Girdle's result is that
there is no end to the mathematical
structure.
So is the incompleteness theorem that
you will never be able to complete it.
You'll never be able to get to sort of
ba oh god I'm going to put words in your
mouth base reality. I don't know another
way to say it.
That's right. That's right. That that's
what that girdle theorem seems to be
telling us is that
no matter how complicated the
mathematics is that you know effectively
you haven't even begun yet. When you say
theorem, I assume that means that this
is a math equation that Girdle put
forward and not it's not Girdle's
theory. It's Girdle's theorem.
It's a theorem. In other words, it's
true
that we can't ever know all mathematical
truths.
Interesting. I don't understand math
well enough to even know what to ask
beyond that. So, to me, we're my my
ignorance makes this a miracle. I'm
willing to accept it just to not be
abusive to
I'll just give you a clue about the kind
of thing that he does right
in the math
in the math it's there are these things
called self-referential statements that
cause problems so if I say um
this statement is false
now was that statement true or false
let's let's look if the state the
statement is this statement is false
well if it's false then it's true
true right
but but what then If it's true, then
it's false.
That's right. So, you get these when you
have self-reference,
you get these problems that that pop up.
You know, the So, the the barber of
Seville cuts all and only the hair of
those who don't cut their own hair.
Who cuts the hair of the barber of
Seville?
Right?
All these kinds of things. So, Girdle
was able to take this kind of thing and
and and make mathematical statements
self-referential. And he was able to
create a sentence that said that that a
statement in mathematics that says this
sentence is true within the system but
can't be proved.
So he was able to construct that that's
what so he actually has a theorem. He
proves and and then he shows that no
matter so if even if you add that
sentence in there'll be a new sentence
that's self-referential that and so what
he shows by by this this kind of
structure of course um the the the true
theorem and the true proof is is
incredible there's girdle numbers it's
you have to be not just a mathematician
you have to be a brilliant mathematician
a logician to even understand it so so
it's it's very very few people who
actually understand
but every time you quote unquote solve
it it's self-referential again
that's right it's the self- reference
that's the key so just so there's not in
complete magic. I want to let you know
that based on this notion of a
self-reerential statement, but but the
bottom line is it shows that there's an
endless
in principle endless possibility of
exploration of mathematical structure
and and since I just mentioned
so just to like almost give it a silly
answer, it's the the mathematical
equation is what's below math and the
answer is math.
It's structure all the way down forever
forever forever. But if consciousness is
fundamental, that means that there's
endless conscious structure. Endless
consciousness.
I feel like that's just something that
says we don't [ __ ] understand. Like
that to me, quite frankly, is as sort of
I just have to accept it as turtles all
the way down. Saying math all the way
down or consciousness all the way down
is the same as saying turtles all the
way down. There's literally no
difference in my limited mind.
Right? So, this is where we're going to
have to come with every scientific
theory, right? Every scientific theory
will have some set of miracles and
that's that bothers me as much as it
bothers you. Yeah,
I don't like it.
But I So when I bump up against that, I
take a pretty and I don't maybe this is
stupid. Enlighten me, but here's here's
where I come up. So I will routinely be
asked if I believe in God. And the
answer is no. I don't believe in God in
any of the ways that people mean when
they say God. But there is so obviously
something that I don't understand that
it is just self-evident to me you would
use the language and maybe rightly so
that it this is a headset and so you
have an intuitive sense that there is
something beyond the headset and you
have no idea what it is and therefore
you just say there is something I don't
understand. Right? So um even here's the
thing that used to [ __ ] with me as a
little kid.
The universe is expanding. Expanding
into what? When you build a house you
build it on land. Right? So the land is
there. The land is on the planet. So,
it's like there's this sense of um to
for it to expand, it has to expand into
something. Therefore, something had to
exist. It was very easy for me to just
go, "Yep, there's something here I don't
understand." The fact that um general
relativity and quantum mechanics don't
play well together. Yep, there's
something I don't understand. Like, I am
I am very okay with just going, "Ah,
there's something I don't understand."
It's it's the the part that gets hard
for me is the it's math all the way down
where it's
it's no longer an acknowledgment of this
is something we just don't understand
and it's saying and now believe that
it's math all the way down and I don't
know that it really matters to be honest
but that that's where I always bump and
go
well yeah there are two big camps on
this one is that we invent math and the
other is we discover it right so when a
new theorem is
published, did the person discover it or
did they invent it? Right? That's that's
one of the big questions. If if isn't it
self-evident that it has to be
discovered?
Well, but if if it is discovered, then
Girdle Girdle's incompleteness theorem
says it is math all the way down, right?
So that's that's that's that's the
thing. But if if it's discovered, I'm
sorry, if if it's invented, then that
points at us and says who who is this
discoverer? Who is this inventor that's
doing all this? So being so and it's
really important for everybody to
understand that that every scientific
theory as we said earlier stops
explanation stops. There's going to be
some place where we say grant me this
please and if you grant me this then I
will explain everything else like
Einstein says grant me space and time if
you grant me space and time I will write
down these mathematics and then it turns
out there's black holes and they're all
interesting stuff we can do GPS because
of Einstein's theory of special relat
general relativity it it gives you all
this but we are granting space and time
now someone else like there's a guy
named Seth Ly who says okay I'm not
going to grant you spaceime I'll start
with quantum bits and quantum gates
outside of space and time. It's just
abstract quantum computational stuff.
And I can show you how to boot up
spacetime, general relativity from
quantum bits and quantum gates. The
curvature of general relativistic
spacetime has to do with the action of
the gates and so forth. So he's no
longer assuming spaceime. He's
explaining it, but now he's asking for a
different miracle. Please grant grant me
quantum bits and quantum gates.
Now you can imagine someone going,
"Well, now I'm going to do better than
Seth Lloyd. I'm gonna explain I'm gonna
do something deeper than quantum bits
and quantum case. That's perhaps what
what Eninar Aani Jimeda is doing at at
you know Princeton.
So maybe he'll get quantum mechanics
emerging from something deeper. But he
will then say grant me what he's asking
for is the amplitude and some other
structures like that. So if you grant me
this amplitude I can give you space time
and quantum mechanics and and so forth.
So, so that's the nature of explanation
and I don't I'm still having to come to
terms with that. I would like to have a
theory of everything and I can only have
a theory of everything except these
assumptions for my theory.
And those assumptions are my miracles.
And so at the foundation of every
scientific theory, there's this moment
of humility.
Explanation stops here. It also stops
one other remarkable place.
Whenever in our theory we have a
probability
that can't be reduced
by greater knowledge. Right? So in a
Newtonian world if I flip a coin in
principle if I knew in detail the mass
of the coin and its distribution and
exactly how I flipped it I could tell
you heads or tails with probability one.
But I don't know the initial conditions
well enough. And so I have to give you a
probability of a half. That's a
subjective probability, epistemic
probability. But suppose
no matter there's a probability that no
matter how much I know the probability
can't completely go away,
then that's no longer epistemic. There's
something more interesting going on.
This was the debate between Boore and
Einstein about quantum mechanics. There
are probabilities that come up there.
Einstein was saying those probabilities
are just our lack of knowledge. There's
no God doesn't really play dice. There's
no fundamental probability going on
there. Boris said no no no. You don't
tell God what to do. These are not
epistemic probabilities.
These probabilities cannot be reduced.
Period. They're probabilities for
everyone including God. Metaphorical
God. Uh, one thing I wanted to ask you
that came to me when I was reading is
the double slit experiment is one of the
weirdest thing in physics for me that it
really messes with me. So, um, for
people not familiar with the double slit
experiment, you take a single photon,
you shoot it through a slit and if
you're not um, measuring it as it
happens on the sort of back wall, you
would see like a bullet mark, right? So,
you fire a single bullet, it goes
through the slit and it hits the wall.
Um, if you
uh sorry, that's if you watch it. If you
don't measure it, then it goes through
like a wave and you get an interference
pattern on the back. You can put another
slit and if you're watching it, it goes
through one slit. If you're not watching
it, then it goes through both slits like
a wave.
Uh, that always just seems so weird.
But if we Well, in fact, I'll I'll ask
does your hypothesis about consciousness
address the issue of the double slit
experiment?
Yes. So, so it will come out, I believe,
that what those experiments are showing
us is that if we assume that everything
is happening in space and time and that
spaceime is the fundamental reality,
then we're going to be confused.
What we're seeing is spacetime is just a
visualization tool we're using for
things that are happening outside of
space and time. And so, they're not
constrained to travel through space and
time. Absolutely not constrained. We're
constrained to see them as though
they're traveling through space and
time. And that's why quantum mechanics
looks so weird.
But why does um observing or not
observing change its state
because
we're creating reality
as we render it.
That's what I was wondering if this is
the look at the moon not look at the
moon.
Exactly right. The moon does. In fact,
Einstein asked one of his colleagues
when they were walking said, "Do you
really believe the moon doesn't exist or
only exists when someone looks doesn't
exist otherwise?" He was talking about
quantum mechanics and
my interpretation of quantum mechanics
is exactly that that that spaceime
itself doesn't exist when it's not
observed and therefore the particles
inside spaceime.
I want to hear you talk about the double
slit experiment. Doesn't that sort it
potentially? Let me ask is that
potentially like a a proof of your
theory?
Well, unfortunately, there are no proofs
in science
there. You
every theory has lots of hypotheses and
auxiliary facts and assumptions and so
forth. And if if your theory doesn't
come out quite right, you don't know
what went wrong in it. And also um even
if every experiment that you've done is
compatible with your theory, maybe you
just haven't been smart enough to think
of the experiment that will take it
down. So So no real serious scientist
would say that any scientific theory has
been proved.
Does it point in the right direction
though? It's like you you bring up a lot
of examples, but that's not one.
Is there a reason? Is there a hole in
that one already that you see or cuz to
me that that is some compelling [ __ ]
like hey you want to wonder or you want
to know if the moon exists when you look
at it or if it's garbage bin uh I keep
forgetting your language garbage
collected garbage
garbage collected when you look away
boom double slit experiment
well so the double slit experiment is
completely compatible with what I'm
saying the reason I don't take it as a
proof is because there are some
physicalists who have the multiverse or
many worlds interpretations of many
worlds interpretations So Hugh Everett
um for example. So what these guys will
say is that um to really understand
superposition and all these weird
quantum things you have to realize that
whenever you make a measurement whole
new universes spin off and all possible
states that are allowed by the quantum
state function, the wave function
um are in are true in some some
universe. And so many
serious physicists believe in in the
many worlds interpretation and related
but but distinct thing of the multiverse
which is a different thing. They they
think that there are multiverses and so
what what is
what is true
is that local realism is false. So local
realism is the claim that
objects in spaceime like say a proton
have definite values of their properties
like position, momentum and spin
definite values
um even when they're not observed
and there's two parts. That's the first
part and that they have influences that
propagate no faster than the speed of
light. We have very very good evidence
to say if if we know anything, we know
that local realism is false. But that
leaves open whether it's the locality
that's false. Things you could imagine
things having influences faster than the
speed of light. So guy named D balm has
a theory in which things have influences
faster than the speed of light or
whether it's realism that's false that
which is what I'm claiming that realism
is false that a particle doesn't have a
position or momentum or spin when it's
not observed because you create it as a
headset element when you observe.
So that's so I say that we know that
local realism is false. I claim that's
the realism that's false but there are
some who can claim it's the locality.
Right. But this is an example of you
saying you have to have a theory that
can be backwards compatible essentially.
That's right. Right. Right.
Makes sense. I'm going to ask you a
super random question because I'm so
curious. You have a wedding ring, so I'm
assuming you're married, so I'm assuming
you love somebody. Yes.
Um
that's that's such a headset experience.
How often are you sort of
in sort of, hey, this really is real and
I'm going to treat it as such. And like
does your wife think this is all like
does she find this decidedly unromantic?
Well, my wife is an artist. She's a very
talented art artist. Her name is
Gerilyn. I call her Jerry.
And she's, you know, she's um not a
scientist and she appreciates my science
and even though she can't really
understand any of the math and I
appreciate her art even though I'm a
monkey looking at Mozart, right? You
know, because I'm no artist. And so we
have mutual respect for the talents, the
complimentary talents, and we sort of
compliment each other. And
but do you just have to turn that part
of your brain off that's like
a lot of time with herusion?
Well, with her, yes, because um she'll
she's happy for me to talk with her
about it um up to a point and and then
we need to do something different,
right? And that's that's healthy for me.
But yeah, do you wanna So one thing that
I find interesting is people that
believe that this is all a simulation
and that if we just find the right
equation, we could essentially exit the
simulation,
right?
And that's always struck me as um
intuitively false that there would be no
way to exit the simulation. Again, this
goes back to what I was saying earlier
about the person who takes their headset
off. They would would just be so
fundamentally different the way that you
would be processing data. There's no
sort of core you. I think like I can't
conceptualize it feels to me unless the
I mean God is this what you're saying
in fact let me ask is this what you're
saying that there is this this uh
conscious agent that I in the headset I
in fact no there is a conscious agent
that is me and the way that I present to
you right now is me with the headset on
but the conscious agent could actually
possibly remain intact when it pulls off
the headset. you addressed this a little
bit earlier um but it's it's now
becoming more concrete in my mind
right it it seems to me quite possible
that conscious agents will continue to
exist even when they step out of this
so I could take the headset off
effectively exit the simulation be like
what the [ __ ] is all this around me but
still have
a sense of emotion and attachment and
love and
dare I say vision
quite possibly in the sense that that
this isn't the only headset so maybe by
taking this
but the headset would essentially be my
brain right so I'd be I'd be out of this
headset and would I necessarily be
putting on another
Well, the headset isn't your brain.
Remember, the the headset isn't your
brain. The brain is just one of the
symbols in your headset
for what I'm using
for all the agents for all the agents
that are combining to form you. So, I I
imagine that it's possible for an agent
to go to a different headset.
Could they go to no headset or would
they be then existing in a realm of pure
math?
Well, and that that I don't know. So, in
the in the following sense, so so it's
it so the part that I think I'm
confident about, but again, we'll see
agents could get new headsets and a wide
variety of them. So, we could really
explore and we could and it could be
very
headset we can shorthand tovel.
That's right.
A different way of perceiving. I can
jump in a bat headset. I can jump in a
dolphin headset
and maybe let go of space and time and
do something different than than space
and time.
My notion of self may migrate in the
process, right? I may keep some of my
emotions, maybe not others. Who who who
knows? I mean,
do you think of taking the headset off
is as an event horizon beyond which we
just cannot possibly even guess?
The
the mathematics that I've got says that
a conscious agent
always can have awareness without
experience.
There is
awareness without experience.
That's right. The math is very clear
about that. So when I write down the set
the space of possible conscious
experiences of conscious I have to write
down what's called a probability space
which is a set of possible conscious
experiences with
I think of awareness as I exist.
Yeah. No I this just awareness without
an eye.
So this would so and by the way I make
no claim to be you know expert in any
like mystical spiritual traditions like
Buddhism or Hinduism or so forth. But
I've I've been told that they do have
this notion um and so I'm not speaking
as an expert, but I've heard that they
have this notion of awareness without
content and that certain meditators
claim to to to be there. And you know, I
I meditate a little bit and I might get
little glimpses of of a notion of
awareness without self without any
particular content. That's the closest I
can get
conceptually to thinking about a
conscious agent without a headset. It's
a a field of pure awareness. But it it
transcends then any emotion, any notion
of self, any specific conscious
experience.
But you've never done psychedelics
because the number one punchline that
people say is, "Yo, you have no sense of
self." Like you dissolve into oneness,
right? Yeah. No, I I I Why don't you I
haven't um I may at some point um you
know where it's legal and so forth so
far.
Well, what's stopped me so far is
well I'll put it this way. I've been
reading the experiences of people. So
I'm benefiting from their experiences. I
talk with with people extensively
actually some very very extensively
about their experiences and so I've I've
studied them. Um there is a price to
pay.
I um
but only in the headset.
But only in the headset. Well Well, and
while I'm in the headset, see, I really
I'm so eager to pursue these ideas. I
think I've gotten I know people say,
"Well, you if you don't do it for
yourself, you can't know what it's
really like to be without a self." Well,
but I can look at the math and the math
actually says, "Yeah, I'm seeing that in
my math." So, I've gotten the insight
that I need there.
Did you listen?
Maybe I will do at some point, you know.
Did you listen to Sam Harris's podcast
about taking five grams of psilocybin? I
think
I haven't heard him do that.
Dude, it's very interesting because I
know you know Sam. Um, you must listen
to that podcast. It is [ __ ]
fascinating. And he was talking about
the fact that um, and I haven't done it
either, by the way. Uh, so and I have a
very clear I am a physicalist. Is that
what you call?
Right. Physical.
Uh, so because of that I'm like I am not
[ __ ] with my brain. Like I'm super
paranoid. I do part of me wants to do
it. So if you cut my corpus colossum,
one hemisphere wants to do psychedelics
really badly and the other side is like
get the [ __ ] out of here. We're not
doing psychedelics is a very bad idea.
Absolutely.
So I I'm sort of stuck in this like go
no go scenario. Anyway, Sam
I agree that's when I said there's a
price to pay. That's exactly what I was
that's that's the dialogue.
I'm super paranoid. So Sam said in in
the experience he had a moment where he
he forgot that he did drugs. Like he had
no sense of oh I have taken a drug to be
in the state and I'm simply in the
state. I'm going to be in this state
forever and this is what life is.
And he was like it it is a type of hell
where it is just it's uh now I'm putting
words in his mouth. The feeling I got
and he may have actually said this was
that it was like a form of just
neverending terror, right? And so you
sort of pass through that and he was
like there are other moments where it's
it's never ending bliss and you're like
it's gonna be blissful forever
and it was just the thought of where you
have no concept I have done something to
myself where I have taken a drug and
this is a consequence of that and it
will ultimately wear off. uh that it's
interesting. It is interesting, man. And
I get how that can really shape people's
perception of what is real and like
really shake you loose
because as as I read this stuff and for
anybody that's made it this far, first
of all, congratulations. This [ __ ] is so
deep and so heavy. But the more time
that I've spent with your ideas, the
more I actually I feel like someone is
sort of filing off a callous on the
bottom of my foot. And it's like, oo,
whoa, we're getting to like a different
sensation here. I never realized that I
had uh a perception in, you know, when
you've got the calluses that are like
half an inch thick, it's like you just
forget that that can actually feel
something. So, the more time I've spent
with your ideas, the more I'm like, man,
there really is something here. And so I
the first note I took on you was so
arrogant and so aggressive and I was
like dude does this guy not realize that
he invalidates his own theory by saying
like this is about natural selection
because I didn't understand your whole
thing about math and reason
right
so but then you like you start spending
more time with like [ __ ] I can't just
discredit this
then you start exploring it like I
definitely before I started thinking
about your stuff diehard physicalist
I will pro regression of the mean right
so I'm going to slide back to that
unless I really spend time on this
because it is such a compelling
illusion.
But it it does
I can feel things pushing me at my back
to experiment with psychedelics for
reasons like this where I yes I could
meditate for the next 40 years and maybe
get to the point where I could have one
of those experiences of what it means to
be aware without a sense of eye. I I
literally can't imagine what that is
right now. I can imagine blanking out,
right? I can't imagine having a sense
that there is awareness without
um me being inserted somehow into that.
That's right.
Um so it would be really fascinating to
very quickly be ejected out of you know
my normal state of consciousness and
into this.
And part of part of what I promised
myself I would do in this interview is
to really figure out like why the [ __ ]
does this all matter? Like you and
that's why I'm like I'm dancing around
this idea of like you love your wife.
like there there is a realm in which
you're like the headset is pretty rad
and it's given me amazing [ __ ] and I'm
really having a hard time like actually
stepping outside of the headset
and so so much of my life is predicated
on the headset it may not even be
possible to retain
myself which I value I'm assuming right
so I value myself I value this
experience I [ __ ] value my wife
that's the one that really scares me
like
I get so much out of my wife like I've
imagined my wife and I actually had this
conversation one time
like, "Hey, a magic genie shows up. What
do you wish for?" And she was like,
"Please don't wish for super
intelligence." And I was like, "Why?"
Like, that's such a rad option, right?
To be the smartest human that ever
existed. She said, "If you did that,
you'd no longer be in love with me."
And I was like, "Fuck, you're actually
right
because if you're you wouldn't love any
human anymore. Like, I love my dogs, but
not like I love my wife because there's
such a gap in in how we can relate to
each other." Right?
So, when I think about like
stepping outside of the headset, man,
you're giving up everything that you
value. And that's like really,
really trippy. Now, I admit if you told
me, "Hey, Tom, here's like a pill you
can take and it's going to give you a
little peek outside the headset." I'd be
like,
"Right,
I'd have to do it. I'd have to take a
peek. I'd have to see like what it was."
Because I don't believe that drugs are
necessarily punching through to some
truth. I don't find myself like super
compelled to do it. But the idea of
looking beyond is both exhilarating and
terrifying because of that loss of the
things that I I am so invested in.
Yes. I I'm on the same page with you and
and it's you know interaction with my
wife is one of the greatest pleasures is
you know makes life meaningful and my my
daughter and I've got three grandkids
and and you know my son-in-law was and
you know my students
son-in-law made the list. Not bad.
Yeah. Absolutely. Yeah. Yeah, Jay is a
great guy and and in this case I'm
interacting with other conscious agents
and I'm benefiting. So I'm now going
into my theory. This is now it seems
impersonal but but I'm going back to the
theory within the framework of the
theory. There's a dynamics of conscious
agents is is as conscious agents
interact that they they learn they get
new comprehensions and they create new
agents. So what we're doing in this
headset in terms like we're interacting.
There's something new that's we're both
growing. We're both learning from this
experience. We're we come away from it
different. And that seems to be part of
it. If this kid in the candy store
theory of consciousness is on the right
track, then we're experiencing it right
now. We're kids in candy stores. We're
we're exploring and we're wondering
what's on the what's what's on the next
shelf of candy. Right? That's what we're
saying here when we say, "Well, I'd love
to see what if I take five or what h
what happens what's going on there." So
that may be you know what it's really
about is that it's exploration and maybe
you know in meditation one thing that
does happen
is that you get less and less
grasping
of things that you used to be grasping
about. I I I find that I'm able to to
let go of things that
fears for example, but
pictures of myself, it's it's it's
really in some sense a dismantling.
The the best metaphor I can come up with
is I read sometime that when a
caterpillar goes through metamorphosis,
right, goes into a cocoon, the this the
immune cells of the caterpillar
try to kill and they do kill the cells
that are trying to begin the process of
transformation into a butterfly. And for
a while the immune cells of the
caterpillar there it's a battle and but
eventually the immune cells of the
caterpillar get overwhelmed and then
much of the caterpillar liquefies. Now
that can't be fun, right? I mean mean
liquefaction. I mean that it all the
structures that were everything that you
knew as a caterpillar are turning into
goo.
And no wonder your immune system is
fighting that tooth and nail until your
immune system gets overwhelmed. But
finally the immune system gets
overwhelmed. Most of the structure of
the caterpillar turns into goo and then
the transformation happens. That's what
meditation feels like to me. Which means
it's it's a double-edged kind of thing.
It's it's both extremely painful because
everything that I know and have been
connected to and addicted to is
dissolving. But on the other hand, I'm
realizing wait well that wasn't
absolutely that essential. I thought it
was essential, but it's not. And there's
a new kind of structure that's being
built that I have no idea. A caterpillar
can't figure out what the butterfly is
going to be presumably. And so maybe
that's what it's like to be starting to
change headsets. Maybe that's maybe
meditation is a way of letting go of
some of the the restrictive trappings of
one headset and upgrading, right? And
you're getting the 3.0 now. You just had
the 2.0, now it's 3.0. I don't know. But
but these are the kinds of things I do
want to explore within the mathematics.
And that's why I'm sort of it it all
fits. It doesn't mean it's right, but it
all fits. The kid in the candy store, my
own kid anticipation of seeing what's
next. It it does fit. Maybe that's why I
like the kid in the candy store theory.
Just that's just me. And you know, for
other people, that's that's not what
it's about. But I think all of us do
wonder
about what's next
and why are we here and what is it
about? That's why I'm I mean that's one
reason I do this is it life is very very
short I want to explore and things most
of what I've believed very deeply has
been very deeply wrong. Most of what
humanity has believed very deeply has
been very deeply wrong. We have a very
good we're almost about 100% consistent
in being deeply wrong. We've believed
that spacetime is fundamental.
Almost everybody believes that spacetime
is fundamental. We all believe the earth
is flat. Now a few very advanced
physicists,
Ed Whitten has said spaceime is doomed.
David Gross has said it's and doomed.
Neimar Connie Hamemed is saying spaceime
is doomed. And these guys, especially
Nema, are now really
being adventurous, very, very brave and
saying, "Let's go outside of space and
time into a world where we can't think."
Just imagine what they're trying to do.
We're trying to think entirely outside
space and time.
Like as you said, as as a kid, you're
going what could possibly be on the
other side of space and time? These guys
are saying, not only is there something
on the other side, I need to think
deeply about it. And here's a
mathematical structure in which space
and time, quantum mechanics and
unitarity don't even appear in the
language.
And then I'll show you how our headset,
they don't call it a headset, that's now
me ad living for them. So how space and
time, which I'm calling a headset, how
space and time and quantum mechanics and
general relativity appear
from these deeper structures in which
there's no space and no time. So that
fits perfectly with what what I'm
saying. Now, they have no idea what this
deeper structure is about. And what I'm
up to is I'm actually Nemo gave a a
class at Harvard last fall
more than 20 lectures on for graduate
students on these deeper structures
outside of space and time. I am taking
his class on my own. There's all on
YouTube. So I'm just studying it. I'm
transcribing his lectures, studying them
because I believe that I can show with
my team. I'm mathematicians. So they'll
show that the long-term behavior of this
dynamics of consciousness that we're
working on, what we call the asmtoic
behavior, will give rise to the
structures he's seeing like his
amplitude.
And so the reason why and then the
amplitude, he already shows how to build
up spaceime from that. That way I'll be
able to go all the way from conscious
agents through the long-term behavior
conscious agents through the amplitude
to spaceime. I can show you how the
headset is built. That's my goal. And so
I'm I'm really quite excited. Once I
get, you know, I know enough to be worth
his time, I may talk with him, but but
I'm not going to waste his time until I
know enough. So, I'm trying to figure
out how the headset is built. Once we if
we succeed,
we'll be able to reverse engineer that
headset and the technologies we'll we'll
be able to play with the parameters of
spaceime.
So, it's like suppose you're, you know,
a wizard at Grand Theft Auto and you can
play within the game, do all sorts of
things that people find amazing. That's
great. But imagine someone who actually
knows the source code. They can take the
wizard and they can give him a flat
tire. They can take all the gas out of
his tank. They can make the road
infinitely long. They can do whatever
they want to. They can play with the
very parameters of those. So the wizard
is nothing. All of our science right now
has made us wizards. We're eventually
going to get the source code of the
game.
There's literally a book called Off to
Be the Wizard about somebody who
realizes this is all source code and he
can go in and edit the code.
Wow.
That is uh you should check that out.
Donald, thank you so much. This has been
absolutely thrilling. reading your book,
listening to your interviews, and now
spending time with you is amazing. I am
now going to think of myself as the
caterpillar fighting off the butterfly.
That was amazing. I love that. Uh,
you've liquefied my brain, and I hope
that on the other side is a butterfly.
But, uh, who knows? Thank you so much
for joining us.
It was a great pleasure, Tom. Thank you
so much.
You got it, man. All right, everybody.
That was it. Thank you for joining us.
Conversations with Tom. There it is. If
you made it this far, you are my hero.
Peace out.
[Music]
That's going to be the thing that opens
up so much not only in terms of our
ability to experience the world, to get
out of the narrow, you know, viewpoint
that we're in and and be able to open up
to other things. Um, but it'll also
teach us a lot about how the brain
constructs Qualia and um how we how we
have our experiences in the world.