Traditional Rationality is phrased in terms of social rules, with violations interpretable as cheating—as defections from cooperative norms. If you want me to accept a belief from you, you are obligated to provide me with a certain amount of evidence. If you try to get out of it, we all know you’re cheating on your obligation. A theory is obligated to make bold predictions for itself, not just steal predictions that other theories have labored to make. A theory is obligated to expose itself to falsification—if it tries to duck out, that’s like trying to duck out of a fearsome initiation ritual; you must pay your dues.
Traditional Rationality is phrased similarly to the customs that govern human societies, which makes it easy to pass on by word of mouth. Humans detect social cheating with much greater reliability than isomorphic violations of abstract logical rules.1 But viewing rationality as a social obligation gives rise to some strange ideas.
For example, one finds religious people defending their beliefs by saying, “Well, you can’t justify your belief in science!” In other words, “How dare you criticize me for having unjustified beliefs, you hypocrite! You’re doing it too!”
To Bayesians, the brain is an engine of accuracy: it processes and concentrates entangled evidence into a map that reflects the territory. The principles of rationality are laws in the same sense as the Second Law of Thermodynamics: obtaining a reliable belief requires a calculable amount of entangled evidence, just as reliably cooling the contents of a refrigerator requires a calculable minimum of free energy.
In principle, the laws of physics are time-reversible, so there’s an infinitesimally tiny probability—indistinguishable from zero to all but mathematicians—that a refrigerator will spontaneously cool itself down while generating electricity. There’s a slightly larger infinitesimal chance that you could accurately draw a detailed street map of New York without ever visiting, sitting in your living room with your blinds closed and no Internet connection. But I wouldn’t hold your breath.
Before you try mapping an unseen territory, pour some water into a cup at room temperature and wait until it spontaneously freezes before proceeding. That way you can be sure the general trick—ignoring infinitesimally tiny probabilities of success—is working properly. You might not realize directly that your map is wrong, especially if you never visit New York; but you can see that water doesn’t freeze itself.
If the rules of rationality are social customs, then it may seem to excuse behavior X if you point out that others are doing the same thing. It wouldn’t be fair to demand evidence from you, if we can’t provide it ourselves. We will realize that none of us are better than the rest, and we will relent and mercifully excuse you from your social obligation to provide evidence for your belief. And we’ll all live happily ever afterward in liberty, fraternity, and equality.
If the rules of rationality are mathematical laws, then trying to justify evidence-free belief by pointing to someone else doing the same thing will be around as effective as listing thirty reasons why you shouldn’t fall off a cliff. Even if we all vote that it’s unfair for your refrigerator to need electricity, it still won’t run (with probability ~1). Even if we all vote that you shouldn’t have to visit New York, the map will still be wrong. Lady Nature is famously indifferent to such pleading, and so is Lady Math.
So—to shift back to the social language of Traditional Rationality—don’t think you can get away with claiming that it’s okay to have arbitrary beliefs about XYZ, because other people have arbitrary beliefs too. If two parties to a contract both behave equally poorly, a human judge may decide to impose penalties on neither. But if two engineers design their engines equally poorly, neither engine will work. One design error cannot excuse another. Even if I’m doing XYZ wrong, it doesn’t help you, or exempt you from the rules; it just means we’re both screwed.
As a matter of human law in liberal democracies, everyone is entitled to their own beliefs. As a matter of Nature’s law, you are not entitled to accuracy. We don’t arrest people for believing weird things, at least not in the wiser countries. But no one can revoke the law that you need evidence to generate accurate beliefs. Not even a vote of the whole human species can obtain mercy in the court of Nature.
Physicists don’t decide the laws of physics, they just guess what they are. Rationalists don’t decide the laws of rationality, we just guess what they are. You cannot “rationalize” anything that is not rational to begin with. If by dint of extraordinary persuasiveness you convince all the physicists in the world that you are exempt from the law of gravity, and you walk off a cliff, you’ll fall. Even saying “We don’t decide” is too anthropomorphic. There is no higher authority that could exempt you. There is only cause and effect.
Remember this, when you plead to be excused just this once. We can’t excuse you. It isn’t up to us.
1Leda Cosmides and John Tooby, “Cognitive Adaptations for Social Exchange: Evolutionary Psychology and the Generation of Culture,” in The Adapted Mind, ed. Jerome H. Barkow, Leda Cosmides, and John Tooby (New York: Oxford University Press, 1992), 163–228.
Here is another comment on democracy, and I warn the reader of highly provocative and unorthodox opinions ahead.
Here is a quick example of what I mean when I say that some of the the bloggers here are pickled in our civic religion:
As for liberal democracy, it's clearly an error to assert without further argument that liberal democracy will solve all future problems. But it is not a mistake to say that it is far and away the most successful thing that humans have ever come up with, and so that it is the best framework in which to try to address future problems.
Note that it does not say, "the most successful political system," which is IMO a reasonable assertion, but rather, "the most successful thing," e.g., more successful than our science and technology.
If liberal democracy is more potent or more successful than science and technology, why have there been no significant innovations (or improvements) in that tradition since the Jeffersonian- Washingtonian- Hamiltonian- Franklin innovation? (The civil-rights movement was not a significant innovation IMO, but merely a more consistent holding of the system to promises already made by the original (Jeffersonian et al) innovation. Also, even if we count it as a significant innovation, the makes only one significant innovation in 200 years. The parliamentary innovation is an innovation only if you disbelieve the Jeffersonian argument about the separation of powers and about the advisability of intentionally making it hard to institute sweeping changes.)
Since the last important innovation in liberal democracy, our (Baconian- Galilean- Newtonian) scientific tradition has seen vigorous improvement. Namely, we have seen the Frege- Hilbert- Russell innovation in mathematical logic, the Bayesian- Laplacean- Goodean- Jaynesian innovation (well, the Goodean- Jaynesian part of it anyway: Bayes predates the American revolution and Laplace was 32 when the war ended), the Darwinian- Wallacean innovation, the Lorentz- Poicaire- Einstein innovation, the innovation that is quantum physics, the Williams- Maynard Smith- Hamilton- Trivers innovation in evolutionary psychology, the Kahneman- Tversky innovation.
Similarly, since the last important innovation in liberal democracy, we have witnessed impressive improvement in our technology: the Industrial Revolution, the Eli Whitney- Henry Ford- Frederick Winslow Taylor- Peter Drucker- Deming innovation for running manufacturing firms, the von-Neumann- Turing- Aiken- Backus innovation of the digital computer, the Licklider- Roberts- Cerf- Engelbart- Saltzer- Reed- Clark innovation in packet-switched networking, the Sutherland- Nelson- Engelbart- Berners-Lee- Cunningham- Sanger- Wales innovation in hypertext and communications, etc.
The idea that majority votes tend to produce correct or ethical decisions (either directly or via elected representatives) and the idea that if they do not, then they can be made to do so by promoting social justice, social, racial or economic equality, nonviolence or wider participation in elections and in the public discourse leading up to elections -- that is a really silly and poisonously false idea.
In reality, the capacity to make correct and ethical decisions in our complex modern world is distributed extremely unevenly in the population, and no New Deal or Great Society program or progressive agenda is going to change that fact in any relevant time frame.
At the Singularity Summit, a woman in the audience asked, "I am an artist." How can I participate in the implementation of the singularity (not verbatim)? Well, the answer is, Unless you are an extremely unusually rational artist, you can't. Your best course of action is not to try to add your voice to the conversation. If you want to help, send money or if you seek a more personal involvement, befriend a singularitarian and share with him the knack for pleasure and delightful experience that many artists have.
Is Eliezer going to tell me that my answer to the artist is wrong?
Well, maybe he is. After all, if I understand correctly he depends for his living on donations to the Singularity Institute. Agreeing with me will alienate most prospective donors. They say that it is impossible to convince a man of a truth if his livelihood depends on his not understanding it. Although I believe Eliezer to be much less prone to bias than most, maybe this bias ("livelihood bias"? "bias towards wanting to eat and make the car payment"?) is too much for him to overcome.
Arrogance is not a very nice trait, but does believing oneself to be more rational than most people always entail arrogance? Is there no chain of experiences a human being can undergo under which it is rational to conclude that one is much more rational than the average person?
One important reason liberal democracy and later elaborations involving social justice, equality, nonviolence, universal sufferage have so few thoughtful critics is because those with the skill and knowledge to critique it tend to be employed as scholars, scientists or at least as professionals of some sort, and the ideologues will get you fired from these sorts of jobs if your critique is too successful at finding an audience. Also, it can be quite costly to one's career or one's social standing to create the perception among prospective friends and partners of being recklessly impolitic or dangerously heterodox.
Admittedly, another (quite sensible) reason liberal democracy has so few thoughtful critics is that the two (nondemocratic) political innovations of the 20th Century that were tried on a large scale in the industrial world went disasterously wrong and killed many people. So let me stress that I am not advocating another large-scale experiment in government. I explain the reason for my critiquing of democracy below.
Again, I repeat: most of our received political wisdom is really silly and poisonously false.
It would take a concerted campain of organized violence to correct the problem, however, because you have to divest the ideologues from their power (particularly from the universities and the school system), and if you were to succeed, then human nature being what it is, some new political or religious orthodoxy would take its place in a few generations. Also, there is nothing that the ideologues are likely to do that cannot be corrected over the course of a few human generations. Russia, Central Asia and Eastern Europe were damaged by their 70-year experiment with Marxist-Leninist ideology, but there is no reason to think that they will not recover from the damage in a few generations. I do not mean to dismiss the real suffering and loss of human life and human potential caused by that experiment in Marxism-Lenninism, or of the (less severe) suffering caused by the political system of the Western democracies, but since you and I have only very course control over the political environment, we cannot (without ultratechnologies) prevent it from doing significant damage. It is important not to lose track of the most important consideration, which in this case is as follows: Ever since Kepler, Galileo and Newton, our civilization has seen almost continuous progress in science, technology and the amount of wealth that is available to the average person. As a result, a young prospective rationalist has available leisure hours, bodies of knowledge and tools (like the internet) far better suited to making rapid progress learning rational skill than that available to previous generations. Our political system is not the ideal one to nurture the continuation of this progress (e.g., it has a worrisome tendency to enforce an orthodoxy on social scientists), but it is not too bad, and is probably the best out of all the systems that have been tried. And changing it would be very disruptive and, well, bloody. So, it is probably best to leave it standing as long as it continues to allow scientific, technological and economic progress.
The one exception I know of to the general rule that our silly and poisonous political culture cannot do any damage that cannot be corrected in a few generations is the project Eliezer started, namely, the deliberate creation of an intelligence explosion through AI programming. A mistake there can persist for billions of years and indeed might even persists in ways that cannot be adequately described by the passage of time. Moreover, in 2004 or so, Eliezer authored a document that was titled Collective Volition and is now titled Coherent Extrapolated Volition, which in my humble opinion shows Eliezer to have been as of 2004 too credulous and too enamoured of this idea that important decisions come out better when as many people as possible have a say or a vote or an influence. Note the similarity between the title "Collective Volition" and the fatuous platitude "the Will of the People".
Individuals like Eliezer who have mastered a great deal of science at a young age will tend to have come from loving and saner-than-usual families and to be able to attract saner-than-usual friends and colleagues, so it is possible that they find it hard to imagine fully the mendacity, fatuousness and zealotry of the idealists in our "opinion-making" professions and the ruthless careerism and casual butchering of truth of the realists in our opinion-making professions. It is of course our opinion-makers who create and refine our political culture.
Since I have lived my life on the margins of our society and have depended often on ordinary professionals (in my case, doctors, other health-care providers, a few lawyers) with no particular distinction in rationalist skill, maybe I can convey to the reader just how shoddily ordinary professionals treat evidence and treat hypotheses.
It would serve no purpose IMO to challenge the democratic ideologues on, e.g., Daily Kos or indeed in the vast majority of public forums. It would merely ignite a nasty flamewar and it is unlikely to change anyone's opinion.
In contrast, the presence of Eliezer and young Eliezer wanna-bes on this blog makes it worthwhile for somebody (me if no one else steps up) regularly to criticize democratic political ideals, to try to neutralize the flood of democratic ideology and ideals (as exemplified by the quote above).
Again, I ask the reader to consider the proposition that most of our political traditions are silly, fatuous and false -- the modern equivalent to the Medieval Catholic Christianity. Our political culture enjoys its near monopoly on political opinion, like I said, by making life miserable for dissenters and enacting quite serious and severe punishements on dissenters.
Eliezer has over the last few months presented an accomplished technical explanation of how an individual human being makes good decisions (and how that skill might be improved). His explanations are free from slogans. Everything reduces to the non-mysterious operation of physical laws. When his explanations depend on or refer to received wisdom (about physics or neuroscience, for example), there is no reason to believe that that received wisdom is maintained by the punishment of dissenters, the removal of dissenters from positions of visibility (e.g., academic and journalistic positions), or the denouncing of dissenters as immoral or hateful.
Dear reader, ask yourself, Where can I find a corresponding detailed technical explanation of how my favorite political system makes a good decision? What is the causal mechanism in the political system that produces the correct or ethical decisions? How many decibels of evidence support the hypothesis of the existence of the causal mechanism? Has anyone calculated that quantity? The reader should demand detailed answers free from mysteries and from trite slogans that derive their persuasive power from endless repetition. Whey the voting scheme is run in a "controlled experiment", meaning one in which the voting system is asked to answer a mathematical, scientific, technological or economic question to which the answer is already known, does it in fact produce the correct answer?
When the original comment was posted, those chapters of HPMoR were probably not written yet.
Also, spoilers are often written in ROT13 around here. (https://rot13.com/)