I found pair programming pretty useful when starting a new project from scratch, when changes are likely to be interdependent. It is then better to work with, let's say, 1.5x the performance of a single developer on one thing a time, than to work separately and then try to reconcile the changes. Knowledge transfer is also very important at this stage (you get more people with the same vision of the fundamentals).
This generalizes to other cases when there is a "narrow front" - when few things can be worked on in parallel without stepping on each other's toes.
Even more generally, it seems there are three kinds of clear benefits:
1) Less change synchronization (fewer changes worked on at the time).
2) Knowledge transfer (see @FeepingCreature's answer).
3) Immediate, detailed review - probably fewer defects.
There is also a matter of raw throughput (or how much time is required to make a specific change, while the rest of the code is assumed to stay the same, ignoring the cost of syncing with any changes done in parallel). A naive baseline is that a pair has a throughput of a single developer (since they're working on one change at a time). Fortunately, it can be way better, because one person can just focus on the details on the code and the other on the slightly bigger picture and next steps, look up the relevant facts from the documentation etc. This eliminates a lot of context switching and limits the number of things that each developer needs to keep in working memory. Also a lot of typos and other simple problems get caught immediately, so there is less debugging to do. It's not so clear, what all of this stuff adds up to.
I was able to find some studies about the topic, including a meta-analysis by Hannay et al. TL;DR: it depends on the situation, including how experienced are the developers and how complex is the task). It's clearly not a silver bullet and generally it still seems to be a trade-off between person-hours spent and the quality of the produced software.