Is Perfect Prediction Possible?

I want to make a prediction about you: by the  end of this video, you won’t know what to think.   There’s a paradox where you seem to have free  will, while also seeming not to have free will:   Newcomb’s paradox. You may have heard about  it in the Veritasium video on the paradox:   a curious game-like scenario that splits real  people roughly 50/50 into contradicting but   apparently equally-valid strategies. Derek: Exactly!   Henry: You don’t want to buck the trend? Derek: You got me! I am a one-boxer, I will do it.   The thing is, the Veritasium video and many  other articles and papers on the paradox   mostly ignore two of the questions I had when I  first heard the paradox, fundamental questions   about the actual science (rather than just  philosophy) underpinning the paradox: first,   is the basic premise even physically possible?   And if so… why does it feel so paradoxical?   The answers will take us through casual  calculus, quantum mechanics, chaos theory,   and wondering if we in fact live in a simulation. First though, a recap of Newcomb's paradox:

imagine something – a being, a supercomputer,  or an algorithm – that’s extremely good at   predicting people's behaviour, and it’s just made  a prediction about you. In a room with two boxes,   one is open and has a thousand dollars inside, the  other is a closed jackpot box that might contain   a million dollars or might contain nothing. You  are allowed to choose either just the jackpot box   (worth either zero or a million dollars), or both  boxes (together worth either a thousand dollars or   a million plus a thousand dollars)… the catch  is that the predictor-being-algorithm-thingy   is set up in a way that penalizes being  greedy, and before you entered the room,   it only filled the jackpot with a million  dollars if it predicted you’ll choose only   the jackpot and not also the thousand dollars.   If it predicted you were going to choose both   the jackpot and the thousand dollar boxes,  then the being left the jackpot box empty.    The set of possible outcomes is this: if the being  predicts you’ll take just the jackpot box and so   puts a million dollars in it, either you take  just the jackpot and get a million dollars, or   take both boxes, plus a thousand  dollars. And if the being predicts you’ll take   both boxes and puts nothing in the jackpot, either  you take just the jackpot and get nothing, or take   both and get a thousand dollars. But remember  - the being is very good at predicting: it’s   made this same prediction many times before, with  many different people, and is almost always right.

The supposed reason it’s a paradox is this:  There are convincing arguments that tell you   to take just one box, and to take two boxes. One-boxers say that if you choose two boxes,   the prediction-being will have anticipated  it, placed zero dollars in the jackpot box,   and you’ll end up with only a thousand  dollars. And, their argument goes, if you   choose just the jackpot box, the prediction  being will have anticipated that, and put a   million dollars into it. One-boxers think you  should choose just the jackpot box, and take a   million dollars rather than a thousand dollars. Two-boxers say that once you’re in the room and   the game has begun, the being has already  made its prediction and the money is already   sitting there in the boxes. No matter what  prediction the being made, whether it got   it right or wrong - take two boxes and you  will walk away with a thousand dollars more   than if you took just the jackpot box. Two boxers  think you should choose both boxes since that’ll   always get you an extra thousand dollars. You can watch Veritasium's video for some   of the usual discussion and arguments that  surround the paradox, and maybe pause here   to have a think about your own strategy.   Lots of papers articles declare that these   two analyses are more or less equally valid and  focus on arguing which one is worth adhering to.    But there are scientific tools that allow us to  tell if the setup is even possible, and if it is,   whether you should one-box or two-box. Causal calculus is one such tool. Causal

calculus looks at statistics, not just  to infer correlations between things - it   looks at statistics combined with knowledge of  the real world to infer causality. Here’s how.   We know that the being’s prediction and people’s  decisions are tightly correlated. This is clear   because in the past, the being correctly  predicted everyone's decision almost every   time. Causal calculus tells us that a strong  correlation like this can’t occur by chance,   and it means - causality wise - either the being's  prediction must cause your decision, or your   decision must cause the being’s prediction,  or they both must have a shared common cause.   The timeline of the setup – which is that the  being predicts, then you decide – tells causal   calculus that your decision can’t directly  cause the being’s prediction, because that   would require retrocausality, AKA time travel. And the being's prediction can’t directly cause   your choice, because after deciding whether or  not to put the money in the jackpot box, the   being is kept entirely isolated from you. Which leaves only one option: the being’s   prediction and your choice must have a shared  common cause. Something both causes the being   to place a million dollars in the jackpot  box and causes you to choose just one box,   or it causes the being to place nothing in the  box while causing you to choose both boxes.   This simple causal calculus analysis tells  you the best strategy is to take just one box,   though it’s also maybe not really your choice  because something in the past (like, maybe,   watching this video) convinced you it’s the only  way to get the jackpot, and the predictor knew   that and therefore put the jackpot in the box. What makes Newcomb’s paradox so bizarre and

divisive is when you introduce another  powerful tool of causal calculus:   the power of intervention. Imagine a spectator  sitting in the room with you who saw whether or   not the predictor-being put money into the jackpot  box. From the spectator’s perspective, it’s an   obvious choice: no matter what the predictor did,  regardless of whether or not there’s money in the   jackpot box, there’s still a thousand dollars in  the other box and the spectator will be thinking   to themself “I hope they take both boxes! ” To take it further, if the spectator intervened   and made you take two boxes/took the two boxes  for you, you would always benefit, because you’d   always get an extra thousand dollars. Causal  calculus represents intervention by deleting   the causal relationship between the common cause  (whatever it is) and the choice of boxes, which   means that the box-choice is no longer correlated  with the prediction about what you would   choose - the intervener is now doing the choosing,  and they’re free to be as greedy as they want.   This intervention argument has convinced  adherents to causal calculus that the only   rational decision is to choose two boxes, because,  they say, once you’re in the room, you are free to   decide whatever you want - that’s the whole point  of free will! You can intervene in your own life.   But the intervention argument shouldn’t convince  you to take two boxes: because that would assume   that you… are not you - the point is, if  you've made up your mind, another person   can intervene on your behalf, but YOU can’t  intervene on your own decision process. You   ARE your own free decision process, and the being  would supposedly have predicted that correctly.   At least, according to the setup of the paradox.   But that assumes the setup is even physically   possible. We need to see what physics says about  just how accurate a predictor-being could even be   in the real physical world: is perfect (or  close to perfect) accuracy even possible?

We’ve shown that there must be a common cause  that results in both the being’s prediction and   also in your choice, and the most reasonable  common cause is just the physical state of   your past self: in addition to whatever else  it knows, somehow the being must know enough   about past you (and your environment)  to predict what present-you will do. And while many aspects of how the brain works are  still uncertain, we do know that decision making   involves several cognitive processes occurring  simultaneously via a complex web of billions of   neurons: so a small detail, influence, thought,  memory, or timing difference could in principle   tip you to change your mind, perhaps at the very  last second. This complexity is a major problem   for the existence of a predictor being, since  it must therefore be able to instantiate an   extraordinarily accurate copy of you (and your  mind) in order to make reliable predictions. A similar problem arises when trying to  predict the movement of a double pendulum:   if you make a copy, the two pendulums may follow  seemingly identical movements for a while,   until suddenly their motions diverge, because  there was the tiniest error in the copy. The   more accurately you copy the pendulum and  starting conditions, the longer you can   successfully predict the motion before it becomes  unpredictable again, but eventually it will   become unpredictable. And the brain is FAR more  complex and unpredictable than a double pendulum. So the predictor-being would need an incredibly,  incredibly, accurate copy or instantiation of you   in order to predict your choice of boxes with  great accuracy. And if the predictor-being’s   copy of you is really so precise, then a  bizarre reality arises: you can’t actually   know whether you are you or whether you are  just the simulated experience of the copy,   because the copy must be so accurate as  to be essentially indistinguishable from   you! And if you’re the copy, your choice is  clear: you should pick just the jackpot box,   because that will put the million dollars  in the real jackpot for the real you! In essence - there’s a chance that your decision  IS directly causing what’s in the jackpot box,   not by going back in time or anything, but  because the decision made by a simulated   copy of you is the source of the predictor being’s  prediction - and you can’t tell which you are.

But creating a near-perfect physical simulation  or copy of you is not just difficult in practice,   it may be physically impossible even in principle.   If the physical processes that go on in the brain   rely on any inherently quantum features  of matter - even just the butterfly effect   perturbation from the underlying quantum fuzz  of atoms - then the no cloning theorem (which   is a physical law of our universe that means  you can’t make a perfect copy of a quantum   state without destroying the original) - the  no cloning theorem prevents an accurate copy   from being made, and so the predictor-being  couldn’t make an accurate enough prediction. So quantum mechanics destroys  the setup of Newcomb’s paradox.    But it can also save the setup of the  paradox! There’s another quantum mechanism   by which Newcomb's paradox could perhaps  be physically possible, and which even   provides an answer for the choice you should  make: the mechanism is quantum entanglement. If, somehow, the predictor-being were able  to quantum mechanically entangle your choice   with the amount of money in the jackpot  box, they could achieve perfect accuracy. If (and it’s a big if) they could put the quantum  states of your choice and the contents of the box   into a quantum superposition of “choose both boxes  and the jackpot box is empty” and “choose one box   and it has a million dollars”, then, to an outside  observer, the superposition must collapse into   one of the two possibilities: either the player  chose only the jackpot box and it has money in it,   or they chose both boxes and the jackpot box  is empty. Or if you subscribe to the many   worlds interpretation, these two possibilities  happen on different branches of the multiverse. Either way, entanglement means that from the  player’s point of view they can make their choice   100% freely, and the prediction can still be 100%  correct. It’s similar to how when you repeatedly   measure pairs of entangled electrons, the first  electron measured in each pair will appear to   have randomly “chosen” to be spin up or down,  and the second one will equally appear random,   but when you compare the two, it turns out  their spins were always the same. [Screen note:   Quantum causality shows us that is doesn’t matter  which spin is measured first: they will appear   fundamentally random and always be correlated] In short, quantum entanglement is a viable   way where a Newcomb-like setup can be achieved  (albeit so far only with microscopic particles),   and in this case, the only way to get  a million dollars is to choose one box,   because those two outcomes are entangled  together - it's not causation, it's correlation. Well, actually physicists have proved  that for quantum entanglements like the   one we’re discussing, the casual  calculus analysis breaks down,   and you need a quantum theory of causality!   Which is too much for us to get into here. Or maybe it’s just impossible  to set up entanglement between   your brain and the contents of the jackpot box…

So… what do I think? Is a game like  this possible? I actually think it is,   for reasons we’ve completely ignored until  now: basic probability and human nature. A   100% accurate prediction of all participants is  probably physically impossible, but a very good   prediction seems totally reasonable. You don’t  need to be a magic-all-knowing-being to predict   people with super high accuracy: you just need  to get almost everyone to do the same thing.   Say I guess that a die roll will land on either  a one, two, three, four, OR a five, then I’ll   be right around 83% of the time! It’s a property  of probabilities that if a thing is very likely,   you don’t have to be good at predictions: just  choose the likely thing and your accuracy will   be high, because it will be equal to the  high probability of that likely thing. And that’s where human nature comes in:  we know that people can be primed or   misdirected to make choices in one direction  or another, maybe by the design of the game,   the wording of the explanation, or even  the color of the room they’re in! Like,   imagine if the predictor is a five-year-old kid  who predicts every time that people will take both   boxes. Most people, when faced with real boxes  containing real money, would guess that a five   year old would be bad at predicting and likely  doesn’t have one million dollars to give away,   so the vast majority people would choose two  boxes, fulfilling the 5-year-old’s prediction   that they’ll take two boxes, and making the  5-year-old a super-accurate-predictor-being! This   is an implementation of Newcomb's paradox that has  the same statistics but is much less paradoxical. In short, I bet it would be quite easy to  engineer a real-world scenario in which a   majority of participants choose the same thing -  not by accurately predicting the behavior of any   individual, but by understanding human nature. And  by getting a majority of participants to make the   same choice, you can achieve falsely high accuracy  just by always predicting the most common choice.

So what should you do if you actually find  yourself in the situation of Newcomb’s paradox,   choosing between the possibility of a million  dollars or a thousand or both or none at all? If   the predictor being really is a vast intelligence  capable of remarkable prediction, then causal   calculus tells us that you should always pick one  box, while an intervening observer should always   pick two boxes on your behalf. Simulation theory  says you should pick one box (because you might   be in the simulation). Quantum no-cloning says  that a perfect predictor-being is effectively   impossible and maybe you should greedily pick two  boxes, while quantum entanglement leaves open the   door for perfect prediction, in which case you  would pick one box. Whether entanglement like this   would be possible to set up is unclear, though.   And if a random five-year-old is providing the   money and doing the predicting, everyone should  pick two boxes, which allows the five-year-old   to falsely achieve incredible accuracy. So have you made up your mind about Newcomb’s   paradox? Would you choose one box, or two? Or are  you still undecided? I predicted that by the end   of this video, you wouldn't know what to think.   So tell me - how accurate was my prediction?   Derek: So here are some thoughts that I  was scribbling down as I was watching…

OK, so more from Derek in a second, but first,   instead of putting money into boxes for  strangers - why not support the creation   of more MinutePhysics videos like this one?   If you’ve enjoyed watching my videos, please   consider supporting me on Patreon and help make  MinutePhysics sponsor free - thank you so much!   And now back to Derek’s thoughts. Derek: It is my experience with this

problem that most people have a gut or instinctual  response. And my sense is that they don't really   change that over time. I’m such a one-oxer. I  don't need to maximize every dollar. I don't need   to walk out of there with like, look, I got the  most that possibly could have been got! The prize   is the million, the thousand is the distraction.   Maybe the prediction is much simpler than it’s   made out to be here in that you don't need a  full simulacrum of them and their environment   their experiences, which is obviously impossible,  so maybe the question of, what are they going to   do is actually way simpler than we think it is. Henry: Yeah, that's what that's literally what   the next section of the video is about is about.   Which I agree with. There is plenty of ways around   needing to perfectly clone somebody to predict. Derek: I do like the cleverness of bringing   entanglement in there, which would perfectly  determine the outcomes. Is the human brain a   coherent quantum state? Like it probably isn't.   And so that would be the problem. If all we were   doing was treating electron states as choices or  something, then obviously you could perfectly get   this to work. Which is something interesting  and I hadn't thought about that for sure.