Meaning of the Wave Function

[Lee Smolin]

The real ensemble formulation of quantum theory

Lee Smolin

I would like to describe an approach to the quantum measurement problem I invented a few years ago, called the real ensemble formulation. I am uploading the pdf of a talk, viewers who want to watch a very similar talk may look at either http://pirsa.org/13050072/ or http://pirsa.org/11050022/. The content of the talk is also in two papers: 1104.2822, 1205.3707

The real ensemble formulation is an example of what I would like to call a dynamical solution to the problem of quantum foundations. These are approaches that solve the measurement problem by altering the dynamics of quantum theory.

Dynamical approaches seek to solve the measurement problem by the discovery that the Schroedinger equation is an approximation to, or a truncation of, a deeper dynamical theory. For these approaches the distinctive features of quantum dynamics-especially it’s linearity and time reversal synmmetry-are problematic. Taking inspiration from the fact that every other instance of linear dynamics in the history of physics turns out to be an approximation to a more exact non-linear equation, these approaches take as their goal the derivation of the Schroedinger equation from a deeper, nonlinear dynamics. 

For the dynamical approaches the key feature that distinguishes the quantum ‎world is non-locality. The lesson of EPR and Bell for these approaches is that you can have a naive, classical-like epistemology, with the world constructed from beables, if you are willing to introduce a radical non-locality into nature. We see how this can work explicitly in de Broglie Bohm. Note that the dynamics itself has to be non-local-we know this from the the form of the quantum potential for a multiparticle system in Broglie Bohm.

From this point of view, quantum mechanics doesn’t need an interpretation nearly as much as it needs a completion.

Dynamical approaches to quantum foundations include collapse models (spontaneous and not), de Broglie Bohm, Vink, Nelson and other non-local hidden variables theories. 

I’ve been studying dynamical solutions to quantum foundations since around 1980.

From then until 2006 I worked with theories whose beables are large matrices, which evolve according to deterministic dynamics. The classical variables were the eigenvalues of the matrices, the matrix elements themselves were the “hidden variables.” When the matrices were large I could extract stochastic differential equations to describe the dynamics of the eigenvalues. These, I could show, reproduced Nelson’s stochastic quantum dynamics and hence, I could define a wavefunction, as a function of the eigenvalues, that satisfied the Schroedinger equation to leading order in 1/N (for N X N matrices).

In 2006 I wrote a paper, arXiv:quant-ph/0609109., summarizing the strengths and weaknesses of these approaches and began looking for a new approach.

Also, in 2006, I began collaborating with a philosopher, Roberto Mangabeira Unger, on the implications of the hypothesis that laws of physics evolve in time. This was an hypothesis I had introduced in my 1992 paper on cosmological natural selection. This work resulted in two books, Time Reborn (2013) and The Singular Universe and the Reality of Time (in press, CUP), the latter written with Unger, as well as several papers: (a summary is arXiv:1310.8539.) This work had a profound effect on my thinking about quantum foundations, which may be apparent from the opening slides of the talk. The key new idea is that a viable cosmological theory has to differ in several aspects from theories that describe small subsystems of the universe. One then must see the task of completing quantum mechanics as part of the problem of finding that viable cosmological theory.

One consequence of the work with Unger was to suggest the invention of two new dynamical approaches to quantum foundations, the real ensemble formulation and the principle of precedence. These are the subject of this talk.

[Lee Smolin]

The file of the talk is attached here:

[Lee Smolin]

I am having difficulty uploading the pdf of the talk, but interested participants can download the pdf of a similar talk from http://pirsa.org/13050072/

[Quantum Speculations]

Hi Lee,

In your real ensemble interpretation, a quantum system is one of N similarly constituted systems in the universe, which have been prepared in the same state and are subject to the same external forces as they evolve.

Since an external potential is usually classical and generated by a macroscopic system, this seems to require that there are also N similarly constituted macroscopic systems such as our human beings.

Thus, your interpretation seems to imply that there exist many other human beings living in other places of the universe.

Moreover, it seems that each of the N similar macroscopic systems also has a quantum wave function according to your interpretation, and their behaviors will be not classical.

Are these understandings right? Could you explain a little more detail about your interesting new idea?

Best,
Shan

PS. Since the maximum file size allowed is 2148 KB, you will be unable to upload a larger pdf file. Sorry for the inconvenience.

[Lee Smolin]

Dear Shan,

Thanks for your interest and question. You raise an important issue which is the definition of the ensembles corresponding to the quantum state. These are defined to consist of similarly prepared systems with similar external environments and the same constituents. The crucial issue, which I do address in section 6 of the paper as well as the talk, is what is the meaning of “similar” in this definition.

Clearly “similar” cannot mean exactly identical, or there would be only ensembles with one element. Each atomic system is in a slightly different external environment. (Because the world is complex enough to distinguish different events by their environments.) So I am relying on a notion of approximate similarity of environments. However, I don’t specify this notion, and hence leave it informal. Why? Because the assumption is that QM is itself an approximation to another theory, suitable for descriptions of small subsystems of the universe. Notions such as “similar environments” can be expected to be quantified in that cosmological theory QM approximates.

However, we can safely assume that two atoms can be in similar environments even if one is in a room with an experimenter and the other is in some natural environment, far from living things. So this theory does not in any way require any assumption about aliens.

Thanks, Lee

[Robert Griffiths]

*Griffiths questions
Dear Lee,

I confess I had not looked at any of they ideas you presented before this
morning, and then I have only had time for limited reading, so some of the
following may be off base. But let me go ahead anyway and ask some questions
and raise some issues.

Can you say a bit more about what is driving your ideas? Obviously there is
some discontent with quantum mechanics as currently formulated, but what is it
that you think you can improve on? Many of us consider the measurement problem
of quantum foundations, the inability to describe the measuring process in
fully quantum terms, a real scandal, but you seem to be multiplying the scandal
by multiplying the number of measurements without explaining how they work.
Nonlocality is bothersome to me, and I think it was to Einstein, because how
can you do decent physics without concentrating on some small system you hope
you can understand without worrying about the rest of the universe? I gather
that you are either content with nonlocality or consider it a blessing, which
seems surprising in that this at once raises questions of how to reconcile
things with special relativity. (I should not conceal the fact that I am an
advocate of consistent/decoherent histories, which I consider a local
interpretation of QM, and in this sense much to be preferred to Bohmian
mechanics.) If you don’t want a deterministic universe with time invariant
laws, then the easiest way to move back a bit from classical mechanics is to
assume a probabilistic dynamics with time invariance of the laws of
probability, and this is how I understand quantum mechanics–do you have any
objections to such an approach, and doesn’t it agree with everything we know at
present from experiments? To a very good approximation the microscopic laws of
physics as we now understand them are invariant under reversing the sense of
time, and we can invoke initial conditions to explain macroscopic
irreversibility, but you seem to be throwing all of this away. So I am
perplexed where your motivation lies.

Bob Griffiths

[Lee Smolin]

Dear Bob,

Thanks enormously for your interest. Let me answer your questions in sequence:

“Can you say a bit more about what is driving your ideas? Obviously there is
some discontent with quantum mechanics as currently formulated, but what is it
that you think you can improve on?”

I am driven by the now more than 40 year failure to go beyond the standard models of particle physics and cosmology, as well as the problems of quantum gravity, and the measurement problem in quantum foundations.

In addition, I am deeply curious about two questions:

What chose the laws of physics?

What chose the cosmological boundary conditions?

To me these are all part of the same package because they have the same source: that we are at the point where we have to go beyond a methodology suited to modeling small subsystems of the universe and invent a new methodology appropriate to doing science at the level of the universe as a whole.

With Roberto Mangabeira Unger we have done a careful analysis which leads us to propose a starting point for discovering how to extend the laws of physics to cosmology. The full argument is in our forthcoming CUP book:

http://www.cambridge.org/us/academic/subjects/physics/history-philosophy-and-foundations-physics/singular-universe-and-reality-time-proposal-natural-philosophy.

A summary is my essay Temporal Naturalism: arXiv:1310.8539. The popular version of the argument is in my book, Time Reborn.

“Many of us consider the measurement problem of quantum foundations, the inability to describe the measuring process in fully quantum terms, a real scandal.”

Of course, I agree. But from my present point of view the measurement problem has to be seen as part of the larger problem of finding a proper framework for cosmological theory. Quantum theory as presently formulated can only make sense as a theory of small subsystems of the universe and the puzzles it raises can only be resolved by the construction of an adequate cosmological theory (by which I mean a theory that can also explain the origin of the laws and initial conditions.). Note that this is not a casual assertion, the full argument is a detailed case for this conclusion.

“…but you seem to be multiplying the scandal by multiplying the number of measurements without explaining how they work.”

Please rephrase, I don’t understand this question.

“Nonlocality is bothersome to me, and I think it was to Einstein, because how
can you do decent physics without concentrating on some small system you hope 
you can understand without worrying about the rest of the universe?”

Indeed the conclusion of our argument is that we have to worry about the rest of the universe; we’ve gone as far as we can without taking that on.
“I gather that you are either content with nonlocality or consider it a blessing,”

Yes, indeed, non-locality is also natural in a quantum gravity world where spacetime and locality are emergent from more fundamental causal structures. (As in causal set models, spin foam models, etc.)

“…. which seems surprising in that this at once raises questions of how to reconcile
things with special relativity. “

Indeed a big part of the story is how to reconcile non-locality with the strong experimental evidence for predictions of special and general relativity. Part of this story is a new formulation of GR called shape dynamics.

“(I should not conceal the fact that I am an advocate of consistent/decoherent histories, which I consider a local interpretation of QM, and in this sense much to be preferred to Bohmian mechanics.)”

As I believe QM as presently formulated must be restricted to small subsystems, so must all interpretations of it. The various formulations of QM have their advantages and disadvantages, but I am interested in something different, which is how to discover the right completion of an incomplete theory.

“If you don’t want a deterministic universe with time invariant laws, then the easiest way to move back a bit from classical mechanics is to assume a probabilistic dynamics with time invariance of the laws of probability, and this is how I understand quantum mechanics–do you have any objections to such an approach, and doesn’t it agree with everything we know at present from experiments?”

These are very interesting issues for me, because of arguments we recently developed with Marina Cortes about how the standard paradigm of physics must break down for elementary events-ie for the smallest as well as the largest scales. See
Marina Cortês, Lee Smolin, The Universe as a Process of Unique Events, arXiv:1307.6167 [gr-qc]. in press at Physical Review D.

“To a very good approximation the microscopic laws of physics as we now understand them are invariant under reversing the sense of time, and we can invoke initial conditions to explain macroscopic irreversibility, but you seem to be throwing all of this away.”

I don’t think the appeal to time asymmetric initial conditions explains anything because that itself requires explanation. I follow Roger Penrose who in 1979 proposed that the time asymmetric initial conditions are to be explained by the dominance in the very early universe of time asymmetric fundamental laws, from which the time symmetric laws we know emerge as approximate descriptions of small subsystems. My current work develops this theme. For example, with Henrique Gomes and Marina Cortes we are studying a time asymmetric extension of GR he recently discovered.

The main reason I am convinced the world is fundamentally time asymmetric is because the only way to explain the origin of the laws and initial conditions in a way that implies testible predictions is if the laws evolve in time. The philosopher Charles Sanders Peirce understood this clearly in 1893, it is a lesson we have been slow to learn.

“So I am perplexed where your motivation lies.”

I am very grateful for you giving me the opportunity to try to explain my motivation.

Lee

[Ken Wharton]

Dear Lee,

After reading through some of the PIRSA slides, I think I see the big-picture of what you’re attempting here… and I certainly appreciate that having Hamilton’s Principal Function as the phase of the wavefunction might invoke an interpretation where the distant past is directly influencing the future. (Basically, noting the same strange feature of the H.P.F. that I criticize in my own paper here, and then tackling this strange implication head-on.)

So I’d like to follow up on any connection you might see between the H.P.F. and the action principles that inspired it in the first place (leaving the final endpoint variable, and extremizing the action for every different possible outcome). Does the HPF survive, but the original action principle disappear? In general, I’d like to better understand how action principles fit into the big picture that you’re promoting here. If the laws are evolving in time, then any rule that is extremizing solutions that span a range of time would seem to be a non-starter…?

Also, I’m curious (after reading your response to Bob) why you don’t see any net benefit of putting the explanation of all the observed macrosocpic CPT-asymmetries onto a single boundary constraint on the universe. Isn’t that *some* explanatory improvement over CPT-asymmetric laws? (You implied that such special boundaries are themselves time-asymmetric, but that’s a tricky question; if one reverses the universe around the boundary itself, it might look roughly the same in both directions. CPT-asymmetric laws on the other hand, are asymmetric no matter how you slice them.

I like to note Einstein’s 1905 warning about using an ontology that introduces “asymmetries which do not appear to be inherent in the phenomena”, and apply it to CPT-symmetry. I take this as a warning about introducing time-asymmetries right from the start of theory-building. Since you’re starting off with a time-asymmetric account, I gather you think we have sufficient empirical justification for such a step?

Best,

Ken

[Lee Smolin]

Dear Ken,
Thanks so much for your response. Again, I’llgo point by point:

“After reading through some of the PIRSA slides, I think I see the big-picture of what you’re attempting here… and I certainly appreciate that having Hamilton’s Principal Function as the phase of the wavefunction might invoke an interpretation where the distant past is directly influencing the future.”
I’m afraid I don’t appreciate this point. The phase of the wavefunction does correspond to a beable in the real ensemble formulation, but there is no sense there in which the ‘distant past is directly influencing the future.’ There is such a sense in the principle of precedence but the phase of the wavefunction plays no role there. Perhaps you are able to do something I so far haven’t been able to, which is to combine them.

“(Basically, noting the same strange feature of the H.P.F. that I criticize in my own paper here, and then tackling this strange implication head-on.)
So I’d like to follow up on any connection you might see between the H.P.F. and the action principles that inspired it in the first place (leaving the final endpoint variable, and extremizing the action for every different possible outcome). Does the HPF survive, but the original action principle disappear?”
I find what you are saying very interesting, but these are not things I’ve thought about. Perhaps I should.

“In general, I’d like to better understand how action principles fit into the big picture that you’re promoting here. If the laws are evolving in time, then any rule that is extremizing solutions that span a range of time would seem to be a non-starter…?”

This is an excellent question, I can see how in a causal set picture of the kind Marina Cortes and I introduced the amplitudes for elementary events can evolve in time, but I agree its not clear what this does to the classical limit.

“Also, I’m curious (after reading your response to Bob) why you don’t see any net benefit of putting the explanation of all the observed macrosocpic CPT-asymmetries onto a single boundary constraint on the universe. Isn’t that *some* explanatory improvement over CPT-asymmetric laws? (You implied that such special boundaries are themselves time-asymmetric, but that’s a tricky question; if one reverses the universe around the boundary itself, it might look roughly the same in both directions. CPT-asymmetric laws on the other hand, are asymmetric no matter how you slice them.”

I would put it the other way. For a variety of reasons I’ve come to believe that there is an objective, physical distinction between the past, present and future. Part of this is that there is an objective asymmetry between what we can know or give truth values to in the past, as opposed to the future. One reason out of several I am convinced of this is that the attempts to explain the observed time asymmetry of nature in terms of time asymmetric initial conditions imposed on time symmetric laws appears to fail because the very special initial conditions required are highly improbable or fine tuned, in the space of possible initial conditions.

Once one admits that there is an objective distinction between the past and the future an opportunity opens up, which is that this asymmetry is fundamental and directly explains the several very manifest arrows of time we observe in nature.

“I like to note Einstein’s 1905 warning about using an ontology that introduces “asymmetries which do not appear to be inherent in the phenomena”, and apply it to CPT-symmetry.”

But time asymmetry is all around us, it is absolutely inherent in the phenomena. One arrow of time I dwell on is the electromagnetic arrow: that light brings us coherent images from the past and never from the future. That is, there are only retarded potentials, and no advanced potentials. How is this drastic asymmetry to be explained? As Steve Weinstein has emphasized, this requires drastically time asymmetric initial conditions. My approach is to simply say nature is like that. The real theory must be a time asymmetric extension of Maxwell which only has retarded solutions.
“I take this as a warning about introducing time-asymmetries right from the start of theory-building. Since you’re starting off with a time-asymmetric account, I gather you think we have sufficient empirical justification for such a step?”

From my point of view what is surprising is how long Boltzmann’s story of time asymmetric initial conditions imposed on time symmetric laws has held up dispite the fact that it is inadequate to explain why the universe is still out of equilibrium 13.7 billion years later.

Best,

Lee

[Quantum Speculations]

Dear Lee,

Thanks a lot for your kind reply! I still has one concern. Imagine we prepare an energy eigenstate of an electron or atom in an extremely special and complex external potential. Such a potential can hardly be found in natural environment, and it is arguably that the potential can only be designed and generated by an intelligent being like our human beings. So, it seems that your interpretation still has certain implications for the existence oi aliens like us.

I would like to know what you think about this. Thanks!

Shan

[Lee Smolin]

Dear Shan,

Sure, as I wrote, the theory can be tested by doing exactly what you say: construct a multiparticle entangled state of many degrees of freedom that would be very improbable to have been created naturally. If nonetheless the Schroedinger equation is satisfied that is evidence against the real ensemble formulation. Or you could try to save it with a story about aliens trying to communicate non-locally with us. I personally don’t find such stories compelling but it would probably make a better movie than wormholes. Of course, it could be fun to work out a protocol for this kind of communication.

Thanks,

Lee

[Quantum Speculations]

Thanks, Lee. I see it.

[Ken Wharton]

Dear Lee,

Sorry it’s taken me so long to get back to this thread… I was trying to keep up with the schedule of the other talks. But *this* is my favorite topic.. 🙂

When you write:

” … there is an objective asymmetry between what we can know or give truth values to in the past, as opposed to the future. One reason out of several I am convinced of this is that the attempts to explain the observed time asymmetry of nature in terms of time asymmetric initial conditions imposed on time symmetric laws appears to fail because the very special initial conditions required are highly improbable or fine tuned, in the space of possible initial conditions.”

I’m a little unsure at how “objective” our arrow of knowledge is (isn’t that the definition of “subjective”?), so lets focus on initial conditions.

One issue with cosmological initial conditions is whether “typical” means a near-maximum *global* entropy or a near-maximum *local* entropy. The latter seems to hold for the Big Bang (uniform temperature, etc.; http://arxiv.org/abs/0907.0659 ), and I somehow don’t think that we should judge a boundary by the later dynamics; whether or not higher global entropies were eventually possible (and this is quite tricky in an expanding universe!) almost seems beside the point when discussing the “reasonableness” of a local boundary imposed at any given region near the Big Bang.

Furthermore, most boundary constraints aren’t typically “randomly chosen” from some possibility space; they’re highly special by their very nature. For example, the (spatial) boundary condition on the classical electric field imposed by a perfect conductor is quite special; forcing all the parallel components go to zero is “highly improbable, or fine tuned, in the space of possible boundary conditions.” (to steal your own phrase, here). Does that mean I should doubt that such boundaries exist and look for a spatially- asymmetric version of Maxwells equations near a conductor to better explain this fact?

Now, you may see a very large conceptual difference between spatial boundaries and initial boundaries. I certainly think you see a big difference between spatial boundaries and *final* boundaries. But if your argument that the future is different from the past relies on this assumed difference in the first place, then it’s a circular argument. If we take action principles seriously, the external constraint on the spatial boundary of a spacetime region (in which we’re extremizing the action) works just the same as the initial or final boundary of that same spacetime region.

My above conductor example is quite relevant to your other point:

“One arrow of time I dwell on is the electromagnetic arrow: that light brings us coherent images from the past and never from the future. That is, there are only retarded potentials, and no advanced potentials. How is this drastic asymmetry to be explained? As Steve Weinstein has emphasized, this requires drastically time asymmetric initial conditions.”

I agree that the initial conditions must be “special” (see above); but how is this different from the spatially-asymmetric nature of an E-field to the left of a perfect conductor? I don’t deny that the initial conditions are special; whether they’re “time-asymmetric” or not depends on the point around which you flip time when you compare the time-reversed version. If that point is the Big Bang itself, then the boundary is not really time-asymmetric; just special. (In fact, in almost exactly the same way as the boundary constraint imposed by a conductor is special.) But in the case of a conductor it’s still perfectly *natural*, so it doesn’t seem implausible to me that the Big Bang might find such a special boundary “natural” in quite a similar manner.

” From my point of view what is surprising is how long Boltzmann’s story of time asymmetric initial conditions imposed on time symmetric laws has held up dispite the fact that it is inadequate to explain why the universe is still out of equilibrium 13.7 billion years later.”

Hmmm… Over what timescale would you have expected a global equilibrium to be reached, simply given that it started in a local equilibrium? Wouldn’t it depend on the details of nuclear fusion, the size of hbar and the nonuniformities that eventually lead to gravitational collapse, etc.? Those details are exactly what is unfolding in our universe, so if we’re not at a maximum entropy yet, isn’t the timescale reasonable by definition? (There are other relevant points in that Wallace paper I cited above.)

I have a feeling that our intuitions are about as far apart on the big issue here (the status of past vs the future) as it is possible to be, so I doubt we’re going to come to some agreement here…. But when I look at where our folk-intuitions about time clash with the known physics of time, I conclude that we’re far more likely to err in the direction of thinking the past and the future are fundamentally different, than we are likely to err in the direction of thinking that they are too similar. Those of us who think they’re *precisely* similar, and *all* the asymmetries can be attributed to cosmological boundary conditions, are still a very small percentage of working physicists and philosophers.

Thanks for the interesting conversation!

Ken

[Author]

Posts