Beliefs, after all, are for true things, and if you lose sight of that you will lose your epistemics. If you think only of what gives you an advantage in a debate, of what sounds nice, of what wins you the admiration of your peers, of what is politically correct, or of what you would prefer to be true, you will not be able to actually believe true things.
Paul Graham wrote about this in Persuade xor Discover:
The danger of the [version of an argument intended to persuade] is not merely that it's longer. It's that you start to lie to yourself. The ideas start to get mixed together with the spin you've added to get them past the readers' misconceptions.
I think the goal of an essay should be to discover surprising things. That's my goal, at least. And most surprising means most different from what people currently believe. So writing to persuade and writing to discover are diametrically opposed. The more your conclusions disagree with readers' present beliefs, the more effort you'll have to expend on selling your ideas rather than having them. As you accelerate, this drag increases, till eventually you reach a point where 100% of your energy is devoted to overcoming it and you can't go any faster.
It's hard enough to overcome one's own misconceptions without having to think about how to get the resulting ideas past other people's. I worry that if I wrote to persuade, I'd start to shy away unconsciously from ideas I knew would be hard to sell. When I notice something surprising, it's usually very faint at first. There's nothing more than a slight stirring of discomfort. I don't want anything to get in the way of noticing it consciously.
This also reminded me of the Litany of Tarski:
Draco, let me introduce you to something I call the Litany of Tarski. It changes every time you use it. On this occasion it runs like so: If magic is fading out of the world, I want to believe that magic is fading out of the world. If magic is not fading out of the world, I want not to believe that magic is fading out of the world. Let me not become attached to beliefs I may not want.
Holding that beliefs are for true things means that you do not believe things because they are useful, believe things because they sound nice, or believe things because you prefer them to be true. You believe things that are true (or at least that you believe to be true, which is often the best we can get!).
This is maybe a subtle objection, but I disagree with the implicit rejection of utility in favor of truth being set up here. Truth is very attractive to us, and I think this runs deep for reasons that don't much matter here but on which I'll just say I think it's because we're fundamentally prediction error minimizers (with some homeostatic feedback loops thrown in for survival and reproduction purposes). But if I had to justify why truth is important, I would say it's because it's useful. If truth were somehow not causally upstream of making accurate predictions about the world (or maybe that's just what truth means), I don't think I would care about it, because making accurate predictions about the world is really useful to getting all the other things I care about done.
Yes, there is a danger that befalls some people when they prize utility too far above truth that biases them in subtle and gross ways that lead them astray and actually work against them by making them less serve their purposes when they're not looking, but there are similar dangers when people pursue truth at the expense of usefulness, mostly in the form of opportunity costs. I think we all at some point must learn to prize truth over motivated reasoning and preferences, for example, but I also think we must learn to prize the utility of truth over truth itself lest we be enthralled by the Beast of Scrupulosity.
I think this is an important point. There's a false dichotomy (and a lossy reduction in dimensionality) in "you can believe true things or you can believe useful things". You can and should strive for useful true beliefs. If you're not https://wiki.lesswrong.com/wiki/Making_beliefs_pay_rent , you likely have some useless beliefs, and it's _really_ hard to define "true" for beliefs that don't actually do anything.
You shouldn't believe falsehoods full stop. Falsehoods are not useful beliefs, as they make incorrect predictions. You ALSO shouldn't spend a whole lot of effort on truth of beliefs that don't matter. You should have a LOT of topics where you don't have strong beliefs.
I don't think the "Copybook headings" are a direct reference to truth. Some random googlings suggest that the following is a representative example of those copybook headings, which seem more to me like proverbs and references to old wisdom, than to some core concept of truth:
“Eternal vigilance is the price of success.”
“If wishes were horses then beggars would ride.”
“All is not gold that glitters.”
“Well begun is half done.”
I do think the poem works well for the point you are trying to make, but figured I would provide a bit of context.
Yeah, that's why I said it addressed this point "among others" -- my summary of the poem's message would be something like "There are timeless principles of morality and common sense that are fundamentally true; when what's fundamentally true becomes unfashionable and people believe what's popular or sounds good instead, disaster ultimately ensues."
My post refers primarily to the second part of that message (beliefs are for true things, reject this at your peril) rather than the first part.
One of the core principles -- maybe the most core principle -- of the art of rationality is that beliefs are for true things. In other words, you should believe things because they are true. You should not believe things that are not true.
Holding that beliefs are for true things means that you do not believe things because they are useful, believe things because they sound nice, or believe things because you prefer them to be true. You believe things that are true (or at least that you believe to be true, which is often the best we can get!).
Eliezer referred to this principle as "the void", writing in his "The Twelve Virtues of Rationality":
Before these eleven virtues is a virtue which is nameless.
Miyamoto Musashi wrote, in The Book of Five Rings:
“The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy’s cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him.”
Every step of your reasoning must cut through to the correct answer in the same movement. More than anything, you must think of carrying your map through to reflecting the territory.
Musashi wrote that you must always think of carrying your motion through to cutting; I write, with Eliezer, that every belief and every step in your belief must cut through to knowing the truth.
Beliefs, after all, are for true things, and if you lose sight of that you will lose your epistemics. If you think only of what gives you an advantage in a debate, of what sounds nice, of what wins you the admiration of your peers, of what is politically correct, or of what you would prefer to be true, you will not be able to actually believe true things.
I would like to take the perhaps unusual step of closing with a poem by Rudyard Kipling, which addresses this point (among others) rather well:
As I pass through my incarnations in every age and race,
I make my proper prostrations to the Gods of the Market-Place.
Peering through reverent fingers I watch them flourish and fall.
And the Gods of the Copybook Headings, I notice, outlast them all.
We were living in trees when they met us. They showed us each in turn,
That water would certainly wet us, as Fire would certainly burn:
But we found them lacking in Uplift, Vision, and Breadth of Mind,
So we left them to teach the Gorillas while we followed the March of Mankind.
We moved as the Spirit listed. They never altered their pace,
Being neither cloud nor wind-borne like the Gods of the Market-Place;
But they always caught up with our progress, and presently word would come
That a tribe had been wiped off its icefield, or the lights had gone out in Rome.
With the Hopes that our World is built on they were utterly out of touch.
They denied that the Moon was Stilton; they denied she was even Dutch.
They denied that Wishes were Horses; they denied that a Pig had Wings.
So we worshiped the Gods of the Market Who promised these beautiful things.
When the Cambrian measures were forming, They promised perpetual peace.
They swore, if we gave them our weapons, that the wars of the tribes would cease.
But when we disarmed They sold us and delivered us bound to our foe,
And the Gods of the Copybook Headings said: "Stick to the Devil you know."
On the first Feminian Sandstones we were promised the Fuller Life
(Which started by loving our neighbor and ended by loving his wife)
Till our women had no more children and the men lost reason and faith,
And the Gods of the Copybook Headings said: "The Wages of Sin is Death."
In the Carboniferous Epoch we were promised abundance for all,
By robbing selective Peter to pay for collective Paul;
But, though we had plenty of money, there was nothing our money could buy,
And the Gods of the Copybook Headings said: "If you don't work you die."
Then the Gods of the Market tumbled, and their smooth-tongued wizards withdrew,
And the hearts of the meanest were humbled and began to believe it was true
That All is not Gold that Glitters, and Two and Two make Four —
And the Gods of the Copybook Headings limped up to explain it once more.
* * * * * *
As it will be in the future, it was at the birth of Man —
There are only four things certain since Social Progress began: —
That the Dog returns to his Vomit and the Sow returns to her mire,
And the burnt Fool's bandaged finger goes wabbling back to the Fire;
And that after this is accomplished, and the brave new world begins
When all men are paid for existing and no man must pay for his sins,
As surely as Water will wet us, as surely as Fire will burn,
The Gods of the Copybook Headings with terror and slaughter return!