Even if the money alone isn't enough to warrant the scientist to publish in a no-name journal, the journal would soon stop being a no-name journal because scientists would expect that their colleges want to publish in the journal to get the money. That expectation makes the journal more prestigious. The expectations that other people expect the journal to get more prestigious in-turn will increase it's prestige.
I'm inclined to dispute this point. Setting quite aside the difficulty of setting up such a project, supposing that the money came ex nihilo and we magically caught the ear of prestigious scientists... it is my intuition that our journal would nevertheless fail to gain prestige. I believe that scientists who published with us would be seen as having been "bought", and I expect that this scorn would overpower any demonstrable merit the research or our journal as a whole possessed. "I want to publish in this journal to get the prize money" is a different motivation than "I want to publish in this journal because it has prestige," and I don't think that gap is as easily crossed as you seem to think.
I imagine a possible funder like Zuckerberg who wants to advance medical science and is willing to put down 5 million dollar per year for a decade.
I assume that there's a sizable number of prestigious scientists who dislike the traditional publishers enough that the currently consider publishing in the highly prestigious journals just because they have a high impact factor a bit like selling out.
If you can convince people that the standards of the new journal are actually better than the existing ones that further helps with making the decision to publish in the journal seem virtuous.
Finally I don't see how taking money for publishing instead of taking money from a grant seems more like selling out.
If you can convince people that the standards of the new journal are actually beter than the existing ones that further helps with making the decision to publish in the journal seem virtuous.
I concur with the implication, but that's a very big "if". It's possible that many scientists know that e.g. the CONSORT standards are good, but how many do you think would be able to differentiate between two sets of standards, and determine which one is "better"? In addition, I'm not sure that "virtue" really is much of a factor when deciding in which journal to publish one's research, otherwise we wouldn't see people following the incentive gradients they are.
Finally I don't see how taking money for publishing instead of taking money from a grant seems more like selling out.
One is status quo, the other is novel. Sometimes that's all it takes. I can easily imagine a conversation like the following:
"Hey Kit, in which journal did you publish your recent research paper?"
"Oh, I published it in The Journal of High Standards."
"Huh, I've never heard of them. Why didn't you submit it to The Prestigious Yet Unvirtuous Journal?"
"Well, The Journal of High Standards paid me a few thousand dollars."
"Really? That sounds suspicious. You sure it isn't a scam?"
The idea of money coming from the government to fund scientific research is already well-established (since it's what we do), it naturally appeals to our democratic ideals, and everyone understands the incentive structure involved. The idea of money coming as a reward from a publisher is novel (and therefore weird), and the incentive structure is murkier (and therefore suspicious).
This all said, I'm speaking solely from my intuition regarding how people would react to this situation, and my intuition seems to differ substantially from yours. I'm not trying to convince you that you're wrong and I'm right; rather, I'm trying to signal that there is a wide possibility space here, and I'm not sure why you've picked "offering money will lead to greater prestige" out of it when other possibilities seem to be just as likely, if not more so.
I would argue that this model can also used by governments. If the US government funds the journal, they could limit the money donation to US scientists.
But even today not all grant money comes from government bodies. There are likely researchers who only use government funding and who might be weary of third party funding but today most researchers do take some third party funding so I don't see the big difference to taking a grant from a nongovernment source.
Funding this Journal of High Standards wouldn't be a cheap project
So where is the money going to come from? You're talking about seeing this as a type of grant, but the amount of money available for grants and XPrize type organizations is finite and heavily competed for. How are you going to convince people that this is a better way of making scientific progress than the countless other options available?
The main point of the post is floating the idea. Discussing the idea helps different people to estimate it's value. If people find the idea worthwhile it can spread and maybe someone who does have the kind of money for it decides to fund it.
Prize money helps, but you'd also need to find relevant experts who know enough about each sub-field to tell whether the standards are indeed high. (Usually they are called "judges," but perhaps we could call them "peers?")
It might help to narrow the question: instead of looking for "high standards" (which is vague), the prizes could be awarded based on whether papers already published elsewhere appear to use good statistics. Then you'd only need reviewers who are experts in statistics.
You don't need to look at any paper to decide whether or not the CONSORT standards are high standards. The idea of a standard is that the standard isn't different in every particular case.
In medical sciences peer reviewers don't seem to be interested in doing something simple as checking whether the results that a paper reports matches the one's that were preregistered. They do suggest various changes to papers but there's no standard involved in those suggestions but individual opinion of what style individual reviewers like.
Letting people in the same subfield review leads to an academic climate where papers are only written to be read by other people of the same subfield. The incentives that reviewers in the present system have in the present system are bad. If you ask the average scientists about the quality of the peer review his papers get, they don't think very highly of it.
Our current academical journals are horrible. On the one hand charge a lot of money for access to their papers. On the other hand the fail to uphold the research standards they endorse.
The New England Journal of Medicine for example has endorsed the CONSORT guidelines on best practice in trial reporting. When Ben Goldacre sent him a letter about how out of 23 papers Ben Goldacre analysed only 3 were completely in accord with the CONSORT guidelines, they weren't even willing to print the letter.
The CONSORT guidelines don't require that researchers do fancy statistics instead of using p-values but just require basics like a paper explicitly mentioning when it reports outcomes that weren't preregistered. If the journal would care about high scientific standards it wouldn't be a big deal to uphold the CONSORT guidelines. Unfortunately, the other top medical journals aren't much better.
Why is it hard to establish a new journal that has higher standards? As Yudkowsky writes in Moloch's toolbox that academics have to publish in journals of high prestige and journals happens to be in high prestige when high quality papers are submitted to them. It's a chicken-and-egg problem.
How could we start a new journal that would get scientists willing to submit papers to the journal? If the journal would pay every author of a paper $50,000 for publishing the paper scientists would have an incentive to publish in the journal.
Even if the money alone isn't enough to warrant the scientist to publish in a no-name journal, the journal would soon stop being a no-name journal because scientists would expect that their colleges want to publish in the journal to get the money. That expectation makes the journal more prestigious. The expectations that other people expect the journal to get more prestigious in-turn will increase it's prestige.
Funding this Journal of High Standards wouldn't be a cheap project, but the money that payed to the scientists who publish the papers isn't going to waste. It's like giving a grant to the scientist expect that when you give them the grant you usually ask them what they want to do with the grant money and in this system they can decide for themselves what they are going to do with the money.
It would be a bit like XPrizes but instead of giving the scientists specific goals, it's "Do something worthwhile and then write a high quality paper about it".