Actually, I have a little more to say:
Another way to think about higher-rank density matrices is as probability distributions over pure states; I think this is what Charlie Steiner's comment is alluding to.
So, the rank-2 matrix from my previous comment, can be thought of as
, i.e., an equal probability of observing each of . And, because for any orthonormal vectors , again there's nothing special about using the standard basis here (this is mathematically equivalent to the argument I made in the above comment about why you can use any basis for your measurement).
I always hated this point of view; it felt really hacky, and I always found it ugly and unmotivated to go from states to projections just for the sake of taking probability distributions.
The thing above about entanglement and decoherence, IMO, is a more elegant and natural way to see why you'd come up with this formalism. To be explicit, suppose you have the state , and there is an environment state that you don't have access to, say it also begins in state , and initially everything is unentangled, so we begin in the state . Then some unitary evolution happens that entangles us, say it takes to the Bell state .
As we've seen, you should think of your state as being , and now it's clear why this is the right framework for probabilistic mixtures of quantum states: it's entirely natural to think of your part of the now-entangled system to be "an equal chance of and ", and this indeed gives us the right density matrix. It also immediately implies that you are forced to also allow that it could be represented as "an equal chance of and " where , and etc.
But it makes it clear why we have this non-uniqueness of representation, or where the missing information went: we don't just "have a probabilistic mixture of quantum states", we have a small part of a big quantum system that we can't see all of, so the best we can do is represent it (non-uniquely) as a probabilistic mixture of quantum states.
Now, you aren't obliged to take this view, that the only reason we have any uncertainty about our quantum state is because of this sort of decoherence process, but it's definitely a powerful idea.
The usual story about where rank > 1 density matrices come from is when your subsystem is entangled with an environment that you can't observe.
The simplest example is to take a Bell state, say
|00> + |11> (obviously I'm ignoring normalization) and imagine you only have access to the first qubit; how should you represent this state? Precisely because it's entangled, we know that there is no |Psi> in 1-qubit space that will work. The trace method alluded to in the post is to form the (rank-1) density matrix of the Bell state, and then "trace out" the second system; if you think of the density matrix as living in M_2 tensor M_2, this means applying the trace operator just to the right side of the tensor, i.e. mapping matrix units E_ij tensor E_kl to delta_kl E_ij and then extending by linearity.
You can check that for this example you get the (normalized) 2x2 identity matrix.
You can think of this tracing out process as a quantum version of marginalization. To get a feel for it intuitively, it's useful to consider the following: suppose you are given access to an endless supply of one-of-a-Bell-pair qubits, and you make repeated measurements, what will you see?
It's pretty clear that if you measure in the standard basis, you'll have a 50/50 chance of measuring |0> or |1>. This is the sort of thing a first-timer might pattern match to an equal superposition but that's not correct, no matter what basis you measure in you'll obtain 50/50--this is because
conceptually: you're measuring half an entangled state so the whole point is it can't yield a given state with certainly under measurement
mathematically: the Bell state can be written as
|xx> + |yy> for any orthogonal states |x>, |y> , there's nothing special about the standard basis. So in any measurement, the entangled state has an equal chance of both being x and both being y, so sometime who can only see one qubit will see an equal chance of x and of y
formalism-ly (?): the rule for calculating measurement probabilities, |<x, y>|^2 = <x|(|y><y|)|x> where y is your state, and x is the state whose probability after measurement you wish to know, generalizes obviously to <x|rho|x> for any density matrix; in our case rho is a multiple of the identity, and all states are norm 1, so all potential measurement outcomes yield the same probability.
The point about the Lindbladian is that it's pretty generic for rank-1 states to evolve to higher rank mixed states; it's basically the same idea as decoherence: you entangle with the rest of the world, but then lose track of all the precise degrees of freedom, so you only really see a small subsystem of a large entangled state.
Indeed it's true a given high rank density matrix can have multiple purifications--rank-1 states of which it is the traced out part corresponding to one subsystem--but that's to be expected, in this point of view: if we had perfect knowledge of the whole system, including everything our subsystem had ever become entangled with, we'd use a regular, pure state. The use of a mixed, higher rank density matrix corresponds to our loss of information to the "environment". And yes, the rank of the density matrix is related to the minimal dimension of the Hilbert space needed for an environment to purify your density matrix.
I liked this post a lot!
I can't be the only person who read this and thought of the three major varieties of moral theory: Decision Theory roughly corresponding to naive Act Utilitarianism; Policy Theory roughly matching Rule Utilitarianism or Deontology; and Agent Theory being sort of Virtue Ethics-y.
One thing that would be interesting, is to think about what features of the world would lead us to prefer different levels of analysis: my impression is, the move from CDT to EDT/FDT/UDT etc. is driven by Newcomb-like problems and game theory-ish situations, where your decision process can affect the outcomes you will face; usually this seems to arise in situations where there are agents who have comparable or greater computational power than you. But I'm not totally sure I understand what circumstances would make the Agent Theoretic level more appealing than the Policy Theoretic level.
In the post you appear skeptical of the value of going beyond Decision Theory; while some of the examples used to push FDT and so forth seem at least a little contrived, I think the game-theory-ish stuff feels somewhat compelling to me; and even the contrived examples seem to be telling us something about the value of being able to pre-commit to an action ahead of time.
My final thought: is there any reason to stop at Agent Theory? Are there a class of problems that would push one to consider (what we might call) Society Theories, at a level above individual agents?
Every year, a handful of small children, and visiting tourists, and people chasing stray pets, and people who get lost in the dark, accidentally wander up into the monster's cave and into its mouth.
How big does this number have to be before it's worth whipping up the village to kill the monster by all walking in together?
What if the monster has only recently settled in the mountain, so no wayward children have been eaten yet. The town holds a vote: either we can commit to drilling it into our children, and our stumbling drunks, and our visiting tourists to never, ever, ever, ever go up the mountain, or we can go up as one and get rid of the monster now. How do you think most people will vote?
"It seems like everyone will pick red pill"
-- but in the actual poll, they didn't! So, something has gone wrong in the reasoning here, even if there is some normative sense in which it should work.
My understanding is that 70% of Twitter respondents chose "blue", and I'd expect the Twitter poll was both seen by, and responded to, at higher rates by people with an interest in game theory and related topics, i.e. the people more likely to understand the principles necessary to arrive at "red" as an answer.
Obviously a Twitter poll isn't the real life situation, but a) it is far from clear that "blue"s are committing suicide and b) if you find yourself arguing that a supermajority of humanity is below your intellectual threshold of concern, I think that's a good sign in general to reflect on how much you really mean that.
You can switch back and forth between the two views, obviously, and sometimes you do, but I think the most natural reason is because the operators you get are trace 1 positive semidefinite matrices, and there's a lot of theory on PSD matrices waiting for you. Also, the natural maps on density matrices, the quantum channels or trace preserving completely positive maps have a pretty nice representation in terms of conjugation when you think of density matrices as matrices: \rho \mapsto \sum_i K_i \rho K_i^* for some operators K_i that satisfy \sum_i K_i^*K_i = I
Obviously all of these translates to the (0,2) tensor view, but a lot of theory was already built for thinking of these as linear maps on matrix spaces (or c* algebras or whatever fancier generalizations mathematicians had already been looking at)