The stated aim of this community is a worthy one - “each day, we aim to be less wrong about the world than the day before.” May we all aspire to stay alive to that goal. But one of the biggest dangers we face in this endeavour is assumption conceit – the idea that the basic assumptions underpinning our worldview are absolutely secure.
It is all too easy to become complacent and imagine that core notions framing our version of reality are self-evident truths. We may start to forget the extent to which our perceptions and understanding are shaped by them. It is such a subtly insidious danger that it requires on-going vigilance – even curious, high-IQ individuals are prone to suffer from it. Indeed, one might even argue that a sense of intellectual superiority makes a person even more susceptible.
This risk further increases when we bond together with like-minded individuals in social groups. Opinions tend to become self-reinforcing and a sense of tribal allegiance emerges, wherein it can become taboo to question core aspects of shared group-think. Yudkowsky articulated this danger in the Sequences – “identifying with a tribe is a very strong emotional force. People will die for it. And once you get people to identify with a tribe, the beliefs which are the attire of that tribe will be spoken with the full passion of belonging to that tribe”
So in the interests of epistemic hygiene, we must be prepared to question the metaphysical assumptions of our group. It is a course of action that physicist David Bohm strongly recommended -
One of the best ways of a person becoming aware of his own tacit metaphysical assumptions is to be confronted by several other kinds. His first reaction is often of violent disturbance, as views that are very dear are questioned or thrown to the ground. Nevertheless, if he will ‘stay with it’, rather than escape into anger and unjustified rejection of contrary ideas, he will discover that this disturbance is very beneficial. For now he becomes aware of the assumptive character of a great many previously unquestioned features of his own thinking. [1, p. 42]
Much is to be gained from stepping outside of our mental comfort zone from time to time.
Being committed to trying to figure out “what's actually true (rather than just what you want to be true or just winning arguments)” we must be willing to stress-test core assumptions. For history shows that everyone can be over-confident in their worldview - the danger is not just confined to blinkered theologians. The story of Danny Shechtman and his discovery of quasicrystals provides a good example. Examining x-ray diffraction images of aluminium alloys, he noticed clear evidence of fivefold symmetries in their crystal lattices. But when he published his findings, they were roundly dismissed - the laws of crystallography, born out by observation over decades, were categorical that such a phenomenon was impossible. As a consequence Shechtman was publicly discredited, and mocked by his contemporaries. Even Linus Pauling - the pre-eminent expert in this field, and a very rare double Nobel laureate - declared that “Danny Shechtman is talking nonsense. There is no such thing as quasicrystals, only quasi-scientists.”[2]
However, such certainty proved unfounded: subsequently Shechtman was proved right. Over time, it became apparent that quasicrystals were not even especially rare. Other researchers had encountered them previously, but dismissed the evidence, having been being convinced that they must be mistaken. It was a classic case of confirmation bias; only finding what you expect to see.
But Pauling’s reasoning had been perfectly sound when he dismissed Shechtman’s claims. His miscalculation had not been a failure of rationality, but rather an over-confidence in the solidity of his underlying assumptions. For the existence of quasicrystals represented a serious challenge to his worldview. Pauling had been convinced that a local model of causation could account for all crystallisation, but the formation of a fivefold symmetry could only be explained by way of long-range influences. Yudkowsky was right when he states in the Sequences that deeper rethinks do not come easily - “people don’t think about the real weak points of their beliefs for the same reason they don’t touch an oven’s red-hot burners; it’s painful.”
Instead of recognising the provisional nature of his base assumptions, Pauling sought to trash-talk Shechtman’s credentials as a scientist. Whilst this example does not show us anything that a good rationalist does not already know about human nature, it serves as a clear warning - even the best of us struggle to avoid assumption conceit.
Let us then take hold of the red-hot burners and question two core tenets of belief that form this tribe’s credo – rationality is the best path to truth, and belief in God is irrational. Whilst the former is ‘official policy,’ the latter represents a widely held position amongst a high proportion of users. This is unsurprising given that LW grew out of Yudkowsky’s overtly atheist blog, and his Sequences continue to act as this site's ‘founding documents.’ Therefore, amongst many members of the community there is tacit support for comments like “theism really is the maddest thing you can believe without being considered mad,” or a belief in God is to be regarded as the “canonical example of Wrongness,” or that the low numbers of theist users on the site demonstrates how, if you were, you “probably don’t use thinking to guide the formation of your beliefs anyway so lessons in rationality are a complete waste of time for you.” It seems fair to say that the promotion of atheism has been more than a subtext.
For a significant majority of LW users, the two base assumptions in question are self-evidently the only possible tenets an intelligent, curious and rationally-trained mind can reach. Nevertheless, a failure to scrutinise them risks the pitfalls and traps that rational thought was supposed to help us avoid. Therefore, truth-seekers must be courageous and try imagine what it would mean if they have been wrong all this time.
Therefore, over the coming few weeks, I intend to publish a series of ‘More Wrong?’ posts, for anyone interested in stress-testing rationalism’s weak spots. Sometimes it may be an uncomfortable ride, but when was this site was ever supposed to be about “having pleasant feelings?” So, let us turn to David Lee Roth for words of wisdom - “you've got to roll with the punches to get to what's real.”
[1] D. Bohm, “Further Remarks on Order,” in Towards a Theoretical Biology, vol. 2, C. H. Waddington, Ed., Edinburgh, Edinburgh Press, 1969.
The stated aim of this community is a worthy one - “each day, we aim to be less wrong about the world than the day before.” May we all aspire to stay alive to that goal. But one of the biggest dangers we face in this endeavour is assumption conceit – the idea that the basic assumptions underpinning our worldview are absolutely secure.
It is all too easy to become complacent and imagine that core notions framing our version of reality are self-evident truths. We may start to forget the extent to which our perceptions and understanding are shaped by them. It is such a subtly insidious danger that it requires on-going vigilance – even curious, high-IQ individuals are prone to suffer from it. Indeed, one might even argue that a sense of intellectual superiority makes a person even more susceptible.
This risk further increases when we bond together with like-minded individuals in social groups. Opinions tend to become self-reinforcing and a sense of tribal allegiance emerges, wherein it can become taboo to question core aspects of shared group-think. Yudkowsky articulated this danger in the Sequences – “identifying with a tribe is a very strong emotional force. People will die for it. And once you get people to identify with a tribe, the beliefs which are the attire of that tribe will be spoken with the full passion of belonging to that tribe”
So in the interests of epistemic hygiene, we must be prepared to question the metaphysical assumptions of our group. It is a course of action that physicist David Bohm strongly recommended -
Much is to be gained from stepping outside of our mental comfort zone from time to time.
Being committed to trying to figure out “what's actually true (rather than just what you want to be true or just winning arguments)” we must be willing to stress-test core assumptions. For history shows that everyone can be over-confident in their worldview - the danger is not just confined to blinkered theologians. The story of Danny Shechtman and his discovery of quasicrystals provides a good example. Examining x-ray diffraction images of aluminium alloys, he noticed clear evidence of fivefold symmetries in their crystal lattices. But when he published his findings, they were roundly dismissed - the laws of crystallography, born out by observation over decades, were categorical that such a phenomenon was impossible. As a consequence Shechtman was publicly discredited, and mocked by his contemporaries. Even Linus Pauling - the pre-eminent expert in this field, and a very rare double Nobel laureate - declared that “Danny Shechtman is talking nonsense. There is no such thing as quasicrystals, only quasi-scientists.”[2]
However, such certainty proved unfounded: subsequently Shechtman was proved right. Over time, it became apparent that quasicrystals were not even especially rare. Other researchers had encountered them previously, but dismissed the evidence, having been being convinced that they must be mistaken. It was a classic case of confirmation bias; only finding what you expect to see.
But Pauling’s reasoning had been perfectly sound when he dismissed Shechtman’s claims. His miscalculation had not been a failure of rationality, but rather an over-confidence in the solidity of his underlying assumptions. For the existence of quasicrystals represented a serious challenge to his worldview. Pauling had been convinced that a local model of causation could account for all crystallisation, but the formation of a fivefold symmetry could only be explained by way of long-range influences. Yudkowsky was right when he states in the Sequences that deeper rethinks do not come easily - “people don’t think about the real weak points of their beliefs for the same reason they don’t touch an oven’s red-hot burners; it’s painful.”
Instead of recognising the provisional nature of his base assumptions, Pauling sought to trash-talk Shechtman’s credentials as a scientist. Whilst this example does not show us anything that a good rationalist does not already know about human nature, it serves as a clear warning - even the best of us struggle to avoid assumption conceit.
Let us then take hold of the red-hot burners and question two core tenets of belief that form this tribe’s credo – rationality is the best path to truth, and belief in God is irrational. Whilst the former is ‘official policy,’ the latter represents a widely held position amongst a high proportion of users. This is unsurprising given that LW grew out of Yudkowsky’s overtly atheist blog, and his Sequences continue to act as this site's ‘founding documents.’ Therefore, amongst many members of the community there is tacit support for comments like “theism really is the maddest thing you can believe without being considered mad,” or a belief in God is to be regarded as the “canonical example of Wrongness,” or that the low numbers of theist users on the site demonstrates how, if you were, you “probably don’t use thinking to guide the formation of your beliefs anyway so lessons in rationality are a complete waste of time for you.” It seems fair to say that the promotion of atheism has been more than a subtext.
For a significant majority of LW users, the two base assumptions in question are self-evidently the only possible tenets an intelligent, curious and rationally-trained mind can reach. Nevertheless, a failure to scrutinise them risks the pitfalls and traps that rational thought was supposed to help us avoid. Therefore, truth-seekers must be courageous and try imagine what it would mean if they have been wrong all this time.
Therefore, over the coming few weeks, I intend to publish a series of ‘More Wrong?’ posts, for anyone interested in stress-testing rationalism’s weak spots. Sometimes it may be an uncomfortable ride, but when was this site was ever supposed to be about “having pleasant feelings?” So, let us turn to David Lee Roth for words of wisdom - “you've got to roll with the punches to get to what's real.”
[1] D. Bohm, “Further Remarks on Order,” in Towards a Theoretical Biology, vol. 2, C. H. Waddington, Ed., Edinburgh, Edinburgh Press, 1969.
[2]A. Shtull-Trauring, “A Serious Matter: Dan Shechtman and a Revolution in Basic Science,” 2 April 2011 . [Online]. Available: https://ver2015.presidentsreport.technion.ac.il/a-serious-matter-dan-shechtman-and-a-revolution-in-basic-science-2