Schopenhauer comments on people's vanity, irrationality, stubbornness, and tendency toward rationalization:
If human nature were not base, but thoroughly honourable, we should in every debate have no other aim than the discovery of truth; we should not in the least care whether the truth proved to be in favour of the opinion which we had begun by expressing, or of the opinion of our adversary. That we should regard as a matter of no moment, or, at any rate, of very secondary consequence; but, as things are, it is the main concern. Our innate vanity, which is particularly sensitive in reference to our intellectual powers, will not suffer us to allow that our first position was wrong and our adversary's right. The way out of this difficulty would be simply to take the trouble always to form a correct judgment. For this a man would have to think before he spoke. But, with most men, innate vanity is accompanied by loquacity and innate dishonesty. They speak before they think; and even though they may afterwards perceive that they are wrong, and that what they assert is false, they want it to seem the contrary. The interest in truth, which may be presumed to have been their only motive when they stated the proposition alleged to be true, now gives way to the interests of vanity: and so, for the sake of vanity, what is true must seem false, and what is false must seem true.
But, says Schopenhauer, these very tendencies may be turned around and harnessed to our service:
However, this very dishonesty, this persistence in a proposition which seems false even to ourselves, has something to be said for it. It often happens that we begin with the firm conviction of the truth of our statement; but our opponent's argument appears to refute it. Should we abandon our position at once, we may discover later on that we were right after all: the proof we offered was false, but nevertheless there was a proof for our statement which was true. The argument which would have been our salvation did not occur to us at the moment. Hence we make it a rule to attack a counter-argument, even though to all appearances it is true and forcible, in the belief that its truth is only superficial, and that in the course of the dispute another argument will occur to us by which we may upset it, or succeed in confirming the truth of our statement. In this way we are almost compelled to become dishonest; or, at any rate, the temptation to do so is very great. Thus it is that the weakness of our intellect and the perversity of our will lend each other mutual support; and that, generally, a disputant fights not for truth, but for his proposition, as though it were a battle pro aris et focis. He sets to work per fas et nefas; nay, as we have seen, he cannot easily do otherwise. As a rule, then, every man will insist on maintaining whatever he has said, even though for the moment he may consider it false or doubtful.
[emphasis mine]
Schopenhauer is saying that—to put it in modern terms—we do not have the capability to instantly evaluate all arguments put to us, to think in the moment through all their implications, to spot flaws, etc., and to perform exactly the correct update (or lack of update). So if we immediately admit that our interlocutor is right and we are wrong, as soon as this seems to be the case, then we can very easily be led into error!
So we don't do that. We defend our position, as it stands at the beginning. And then, after the dispute concludes, we can consider the matter at leisure, and quite possibly change our minds.
Schopenhauer further comments that, as far as the rules and "stratagems" of debate (which form the main part of the book)—
In following out the rules to this end, no respect should be paid to objective truth, because we usually do not know where the truth lies. As I have said, a man often does not himself know whether he is in the right or not; he often believes it, and is mistaken: both sides often believe it. Truth is in the depths. At the beginning of a contest each man believes, as a rule, that right is on his side; in the course of it, both become doubtful, and the truth is not determined or confirmed until the close.
(Note the parallel, here, to adversarial collaborations—and recall that in each of the collaborations in Scott [Alexander's] contest, both sides came out of the experience having moved closer to their opponent/collaborator's position, despite—or, perhaps, because of?—the process involving a full marshaling of arguments for their own initial view!)
So let us not demand—neither of our interlocutors, nor of ourselves—that a compelling argument be immediately accepted. It may well be that stubborn defense of one's starting position—combined with a willingness to reflect, after the dispute ends, and to change one's mind later—is a better path to truth.
I opinedthat while I agree that it's possible to concede an argument too early and that good outcomes often result from being obstinate in the heat on an argument and reflecting at leisure later, I think describing obstinancy as "normatively correct" is taking it way too far.
In discussions of rationality, I usually take the word normative—of or relating toan authoritative standard—to refer to the authoritative standard of ideal Bayesian reasoning, often in contrast to what humans do. As Schopenhauer observes, the entire concept of adversarial debate is non-normative in this sense! Aumann's agreement theorem is a mathematical expression of Schopenhauer's dictum that "we should not in the least care whether the truth proved to be in favor of the opinion which we had begun by expressing, or of the opinion of our adversary." And yet people do.
Indeed, so either we take this to mean that any normative claims about how to conduct such debates are necessarily meaningless, or else we allow for a concept of normativity that is not restricted to idealized Bayesian reasoners (which, I must remind you, are not actually real things that exist). Now, I am not saying that we should not identify an ideal and try to approach it asymptotically, but surely it makes no sense to behave as if we have already reached that ideal. And until we have (which seems unlikely to happen anytime soon or possibly ever), adversarial debate is a form of epistemic inquiry we will always have with us. So there must be right and wrong ways to go about doing it.
Retreating to a less censorious venue in which this discussion about social epistemology would be permitted to continue, I replied:
I emphatically agree that as long as there's a recognizably human "we", we will always have adversarial debate with us. Attempts to deny or disparage the necessity of debate in favor of some pipe dream of "collaborative truthseeking" have been and will continue to be a disastrous farce, like communism.
In order to evaluate the truth of a proposition, a cognitive system must evaluate the evidence (or arguments) for and against it. It turns out that humans are particularly adept at rationalization, latching on to a proposition that we want to be true (respectively false), and searching hard for arguments for (respectively against) it. As a result, it turns out to be incredibly productive to pit the efforts of a human who wants the proposition to be true against the efforts of another human who wants the proposition to be false. Driven by vanity to maintain whatever they have said, the disputants will thoroughly explore the space of arguments concerning the proposition, and each painstakingly correct his adversary's errors—probably more thoroughly and more painstakingly than if they were merely curious about the proposition.
Debate works—more than anything else. As long as people are human, it could hardly be otherwise. To simply declare that "we should in every debate have no other aim than the discovery of truth" is about as actionable as delcaring that the economy should function "from each according to his ability, to each according to his need"—and with similar results.
And yet, all this having been said acknowledging that the success of adversarial debate has deep roots in human nature, I think it's important to notice the ways in which human nature is normatively silly. Yeah, okay, I see how it is "that the weakness of our intellect and the perversity of our will lend each other mutual support"—that the need to defend our position gives us motivation to keep working on the argument rather than prematurely folding, and that this isn't even particularly bad for the cause of Truth in the world given that the other guy is going to be doing it, too.
But the perversity of our will is still perverse! That's the opposite of normative!
Sometimes the adversariality of debate is formalized, as in court trials. The plantiff and defendant's counsel each have their functional roles to play within the system, and that role is not to be fully general truthseekers. (That's closer to the judge and jury's job.) In that context, we can say that it is normatively correct that a lawyer should argue for her client's interests, rather than to disinterestedly persue the truth of the case.
But in the absence of such formalized roles, where it's understood that the individual is to be a cog in a larger system—no, it is not normatively correct to refuse to budge in response to counterarguments! When I'm having a discussion, and my interlocutor makes a point against my "side"—or if I notice it independently—I want to acknowledge it right then and there. It's not a natural impulse, but I think it's learnable, and worth learning, as a step towards a better world.
Another commenter suggested that when faced with an obstinate interlocutor, one should troll them with, "But you know I'm right; you just don't admit it."
To which the obvious answer would be, "I know no such thing. I might—or might not—know that your argument seems like it doesn't have any obvious problems. But, as we all know, you can prove anything with apparently-not-obviously-flawed arguments. So who knows if you're right or not? My epistemic state right now is ‘I am right, just like I thought before we started arguing'. If that changes, I'll let you know."
To which the obvious answer would be, "You're lying about your epistemic state. You're not completely epistemically learnedly helpless: if you really thought that reason was so perfectly impotent that apparently-not-obviously-flawed arguments could prove anything with equal ease, you'd never learn anything at all. In order for your belief that you were right before we started arguing to mean anything, it needs to make probabilistic predictions. As long as reason isn't perfectly impotent, that will include predictions about intellectual discussions: being right implies being quantitatively less likely to see people come up with counterarguments that you don't know how to answer than if you're wrong."
Nah, I don't buy that at all. That sort of view suggests a model of reasoning where we can do precise Bayesian calculations on everything all the time (which of course we can't) and where, furthermore, we can perform all updates and think through all consequences near-instantly (which of course we definitely can't).
There is, after all, a difference between "I see no flaws in your argument, right now" and "I see no flaws in your argument, after having thought about it for a while, and looked at the matter from all angles at my leisure, and taken as much time and effort as I think is productive to take to think through all relevant considerations". Your view would only make sense if there were no such thing as computational uncertainty or bounded rationality—if our brains were infinite computers! But they aren't.
So if you make an argument which, apparently to me, proves some point, what is that? A piece of evidence. I will go and consider it, and perhaps I will conclude that I should update my views by some amount in some direction. (Perhaps in the direction of your views, perhaps not.) But that's all.
As for "being right implies being quantitatively less likely to see people come up with counterarguments that you don't know how to answer than if you're wrong", I am not even sure that's true. If it is true, what is the odds ratio? How could we possibly establish it? Merely being right doesn't seem to me to predict the absence of counterarguments I can't easily and immediately answer.
Finally, attend closely to what I wrote (describing my hypothetical response to an apparently convincing argument): not "I have made no update whatsoever, however miniscule", but "I am right, just like I thought before we started arguing". This is perfectly compatible with your view! I could easily say: "Yep, I do indeed think that your apparently convincing argument is a very, very small amount of evidence for your position. Yes, apparently-not-obviously-flawed arguments can't prove anything with equal ease; yes, I expect it to be quantitatively less likely that I should see people come up with counterarguments I can't easily and immediately answer if I am right than if I am wrong. All of that is true, I have taken it all into account; I have made the appropriate update, which is very small; and of course I still think that I am right. Maybe I'll make a larger update later, once I have considered everything carefully, and the status of your argument changes from ‘there are no immediately obvious flaws' to ‘yeah, on consideration, this really is right'. Or maybe I won't! We'll see!"
Before you start arguing with someone, you have some prior idea of how people are likely to respond. (Hopefully because your opinion is not completely uninformed, such that you've read previous relevant discussions and are familiar with the standard arguments on the topic, but more fundamentally because you can't not have a prior.)
When the responses come in, you should have a sense of them being stronger or weaker relative to your expectations. To some quantitative-in-principle (if small and certainly not quantitative-for-humans) extent, the responses being weaker than you expected should give you confidence, and the responses being stronger than you expected should give you pause.
Restated with more concreteness, if the responses are the same nonsense you've already refuted a dozen times before, but coming from otherwise sharp people, that's a sign that you're right (because if there were sensible counterarguments to what you had said, you'd expect these otherwise sharp people to find them).
If the responses are novel challenges that would take you hours to come up with a non-embarrassing counterreply to, and coming from Reddit-tier riff-raff, that's a sign that you're wrong (because if there weren't sensible counterarguments to what you had said, why are these clowns putting on such a good show of it?).
As you point out, if evaluating arguments just naturally takes a long time, then it's a weak sign. But what's been at issue in this thread is whether it's "normatively correct" "to stick one's heels in and be unwilling to budge on a position regardless of reason or argument." I'm saying: no, it's not; the common psychological phenomenon Schopenhauer is describing in which the weakness of our intellect and the perversity of our will lend each other mutual support is normatively incorrect, because we should not in the least care whether the truth proved to be in favor of the opinion which we had begun by expressing, or of the opinion of our adversary.
It makes sense to need time to think things through, but I think it's tendentious to call that stubbornness in the context of the preceding discussion. "Let me think about it and get back to you" is not an expression of being "unwilling to budge".
The problem with this argument is that "let me think about it and get back to you" is useless for continuing a conversation. You say something, I say "let me think about it and get back to you", and... then what? We pause for weeks or months or who knows how long? Then I come back and I say "ok, I thought about it for a while and I still think I'm right". You say "ok, in that case ..." and you make another argument. Or I make an argument and you have to think about it. Pause for unknown amounts of time again. We can't have any kind of discussion like this.
Even if this were somehow epistemically optimal for you and for me (a hypothetical pair of arguers), what of any audience? Or third, fourth, fifth participants? Does it remotely make sense to proceed like this? No, of course not.
What's more, by always doing things in this way, you miss out on the (very likely! very common in practice!) possibility that after many back-and-forths, a long sequence of argument and debate wherein you say many things and I say many things and we each do not budge from our starting positions and do not immediately make any updates on the basis of apparently good arguments from the other, we then both go away and think for a while, and read other things, and talk to other people, and—armed with the many things each person's respective interlocutor said—reconsider our respective positions. If after the first apparently convincing argument made by one of us, the other person had said "let me think about it and get back to you", this would not have been possible.
And on top of all of that, in the latter scenario we would deprive all audience members of the benefit of having an adherent of both of our respective positions represent the complete position, for full exploration and examination, in the course of a temporally compact debate (rather than one which takes place over who knows how long while each participants goes away and thinks after every apparently good argument by his interlocutor). (And this isn't even touching on the practical question of how likely it is to have any given debate participant actually be available after the ostensible thinking time. People get busy, distracted, dead...)
All of this means that (as I say in the linked LessWrong comment) while "let me think about it and get back to you" is appropriate sometimes, it is often entirely correct to continue to hold a position in a debate, even when an apparently convincing argument against it has been made. Certainly I will think about the matter at my leisure, and I may change my mind (and I may even change my mind in such a way as to bring me closer to your view!). But for the purposes of the debate we're having, I will not immediately update, and this is indeed normatively correct. (There are exceptions to this, because it is possible, e.g., to make an argument such that your interlocutor acknowledges that you've made a good point, which he basically already agrees with, but just wasn't thinking about, hadn't recalled, etc.)
The point of "Let me think about it" is that you shouldn't be continuing a conversation if you don't have anything honest to say. If you do have more points to make that haven't already been covered in previous remarks, then by all means, you should continue the full exploration and examination of arguments.
But if an apparently convincing argument has been made, then you don't have a counterargument ready (on that particular point, which may be a small part of a wide-ranging discussion). That's what it means to be apparently convincing. Isn't it better to be honest about that, than to bluff and bluster in order to maintain your position; or to ghost in order to maintain ambiguity about whether the argument was so apparently convincing that you need to think about it, or you just have other duties to attend to?
[Regarding depriving the audience of an adhrent representing each position,] the benefit is in the "full exploration and examination" part, not the "adherent of both of our respective positions" part; it doesn't matter who contributes which parts of the examination. If you make a flawed but repairable argument, and I'm a philosopher pursuing truth rather than a lawyer protecting my client's interests within an adversarial system, it's normatively correct for me to fix it for you if I can.
It's certainly fine (indeed, good) to be honest about the fact that you don't have a counterargument ready on some particular point. However, that's no cause to grant to your interlocutor the validity of the position he's arguing for. Why should you, when you haven't had time to fully consider the matter? "I don't currently have a reply to that point, and I am not (at this time) updating in the slightest on that basis, which is the correct move on my part, because I genuinely don't know whether you are in fact correct, and my current lack of a counterargument doesn't license me to conclude that you are correct" is a perfectly honest stance.
(It's not like "apparently convincing argument that, upon reflection, turns out to be total bullshit" is some sort of esoteric species! On the contrary, it's quite common to encounter such things.)
[Regarding the claim of the benefit being in the "full exploration and examination" part, not the "adherent of both of our respective positions" part:] This seems obviously wrong. In most cases, an adherent of a position will represent it best, both due to familiarity and due to interest. There are of course exceptions, but they generally imply a large mismatch in the arguing parties' intelligence, knowledge, etc. I definitely prefer to see a view explained and defended by its adherents, rather than only by its detractors. Don't you?
[Regarding the the claim about a philosopher fixing their interlocutor's repairable arguments:] If! But why should I assume that you're "a philosopher pursuing truth"? Even if I think that you're attempting to be such, why in the world would I assume that you're succeeding? Isn't this just the old nonsense about "we're all members of a truth-tracking community and should treat each other accordingly"?
In general, I find your whole position here to be inconsistent. You seem to be appealing to a hypothetical scenario where everyone involved is a perfectly honest and perfectly rational philosopher earnestly seeking truth, etc., on the grounds that this would be normative, therefore it is normative to behave in a way that would be appropriate to such a scenario. But given that this scenario is entirely counterfactual, why would it be a good thing to act as if it obtains?
I said, if I'm a philosopher pursuing truth, it's normatively correct for me to fix your broken arguments (as part of the full-exploration-and-examination-of-arguments endeavor). Your trust is neither required nor relevant. [...] [Y]ou can improve your adherence to the ideal unilaterally. It doesn't matter whether the other guy is less rational or less honest. When I admit that he's made a dent in my initial position, I'm not doing it for his sake.
For whose sake are you doing it, then? Can't be for your own—that's unaffected by what you say in a public debate. Is it for the sake of the audience (or other participants)? But I claim that you actually do those people a disservice by doing this. (Indeed, you even do yourself a disservice by doing it!)
Suppose Reality is either A or B, each with prior probability 0.5. Reality generates 20 cards [...] e1, e2, ... e20.
I look at e1, e2 and e3, and they all say "A". That's three information-theoretic bits of evidence that reality is A. That's 8:1 odds, or probability 8/9 ≈ 0.89 that Reality is A.
I encounter you claiming that Reality is B with probability 0.94. "Nonsense!" I cry. By way of explaining, I show you e1 through e3.
You show me e17, e18, e19, and e20, all of which say "B".
If I were certain that you were a psychic all-seeing perfectly dogmatic B-partisan that got to look at all twenty cards and are showing me all of the cards that say "B", then I can infer that all of e4 through e16 say "A" (because if any of them didn't, you would have shown me them). I should end up with posterior odds of 216−4=212=4096:1, or probability about 0.9998 that Reality is A.
If I were certain that you're a philosopher pursuing truth just showing me all the cards you happened to find, then I should count your cards the same as mine and have posterior odds of 23−4=1:2, or probability 1/3 that Reality is A.
Realistically, I expect that you're definitely not all-seeing, and probably some sort of B-partisan but not perfectly dogmatic about it, so my posterior probability should end up somewhere in between the two extreme cases. It would depend on the details. It's not worth specifying the details, because this is just a toy formalism meant to illustrate a qualitative principle of normative behavior. I'm not claiming that real-world arguments are literally isomorphic to cards representing a known likelihood ratio; I just think doing the arithmetic for the cute story about cards is helpful for disciplining the mind and preparing it to confront the normative principle which many find hard to face.
And that principle is, just be honest. That's it! Just be honest! If the card says "B", update on it (where, again, the correct update depends on how you think evidence is being filtered). If the argument seems convincing, say so. (And if you don't see an error now, but you think you likely will on further consideration, then it shouldn't seem "convincing"; you need to recalibrate your sense of convincingness until it's not making predictable mistakes like that.)
I don't think this is a complicated idea. It's not—well, I won't say "it's not hard", because that would be a lie. Schopenhauer knew.
But crucially, as I've been saying, you can be honest unilaterally. The reason debate is crucial for human beings and the "we're all members of a truth-tracking community and should treat each other accordingly" hokum is something only con artists say is because most arguments aren't honest: people have some belief they want to get on the shared map for reasons other than its truth and selectively search for "A" or "B" cards to reveal. There's a fear that if you admit it when the other guy has a point, that puts you at a disadvantage in the competition to get your preferred beliefs onto the shared map (such that even an ideal philosopher who somehow didn't have any motives other than making the shared map more accurate shouldn't do it).
And it would put the philosopher at a disadvantage if the audience can't tell the difference between honesty and dishonesty: if you show all your cards, and the other guy only shows his "B" cards, and everyone treats your revealed cards and his equally, then that would incorrectly bias the shared map towards representing Reality as B.
But I think people can often tell? If I don't "make it a rule to attack a counterargument, even though to all appearances it is true and forcible", that makes it more meaningful when I attack a counterargument that doesn't appear true and forceful, if people are tracking my behavior over time and can infer that I'm not bluffing.
I agree that you should be honest. You should honestly say "I have no reply to that at this time, and I will absolutely not budge at all in my position, because I consider my inability to come up with a reply at this time to be uninformative, and to tell me essentially nothing about whether your argument actually is valid or is instead nonsense, or mistaken, or has some obvious counter which I am not thinking of at the moment". (Or something to that general effect. Certainly you shouldn't lie and say "actually I think that your argument is wrong", or something.)
That would be honest. And correct! Normatively correct!
On the other hand, it would be incorrect to say "hm, I find your argument convincing, so I am updating in the direction of your position". No! Wrong and bad! Very common among rationalists to do this, and totally not the right way to do things.
(Whether it would also be dishonest to say something like that depends on whether you understand that it's incorrect, of course.)
You say: "the correct update depends on how you think evidence is being filtered". Well, how should I know how the evidence is being filtered? Trying to guess how the evidence is being filtered is how they get you. (For a stark example of this failure mode, see Scott Alexander's habit of going "I assume that [the government / the media / third-world dictators / criminals / Sam Altman ] is not telling me the whole truth, but I am going to assume that I can guess approximately how much they're lying, and compensate for their lies, deducing the truth thereby". It doesn't work!) I decline to do this. This is how you get got.
And that's easy mode, even. What about when (as is the vastly more common case) we are not talking about some quantitative "evidence", which you can do math on (including the chance and degree of filtration), but rather an argument, a model, etc.? This is the real problem, you see—not physical uncertainty (where you don't know what is the probability distribution across various possibilities, whether those be on the object level, or on the meta level of "how is this evidence being filtered"), but computational uncertainty, where you can't, in the moment, consider all the implications of your interlocutor's argument, can't fully analyze it, etc. The argument seems convincing, but we know perfectly well that this might just mean that you can't spot the problem right away. Why should you change your view on the basis of "seems convincing"? You shouldn't. It should take "I've given it much thought, and I am actually convinced" to change your mind.
My problem with [honestly saying that I have no reply but will absolutely not budge at all] is that I can't honestly say that, because I don't consider my inability to come up with a reply to be uninformative. Why would it be uninformative?
Often times in such situations I do still think I'm ultimately right for reasons I have yet to satisfactorily articulate. I could say that (and expect to do more work and probably come back later with a satsifactory reply). But I couldn't say "I will absolutely not budge at all in my position" with a straight face. The fact that I have more work to do, and that I can't be surpremely confident in the soundness of work I haven't yet done, seems like a form of probabilistic budging.
(I mean, sometimes it might be. But sometimes not.)
One question to ask in such cases is "informative to whom"? To the ideal Bayesian reasoner who has unlimited computing power and thinks infinitely fast? Yeah, maybe. To you, personally (or to me, etc.)? I really don't see why it must be informative.
But maybe we've gone too long without any actual examples (a sign of this being that we seem to be going in circles). So let's try and make this more concrete. Consider some scenarios:
Scenario 1
I make some argument. Someone makes a counterpoint. The counterpoint is one which I hadn't thought of, but once I see it made, I immediately see that it's correct and that it severely undermines my argument and basically I am just wrong. I am immediately convinced. I no longer think that my previous position was correct. If I had thought of the counterargument myself, I would never have made the original argument. I've actually changed my mind. I say "oh yeah true, good point, you are right".
I have some position, which I am quite sure is right. I am very sure that my view is well-considered and accounts for all possible challenges. I make an argument for my view. Someone makes a counterargument, and it's one which I hadn't anticipated at all. If you had asked me what my interlocutor might say in response to my argument, I would not have predicted anything at all like the response I've received. Their counterargument seems sound. I have canned replies to many kinds of potential rebuttals, but this one just comes out of left field. I have literally never considered anything like this counterargument I've just heard. I am shaken. Obviously, I have no reply to make. I am not convinced of my interlocutor's view (I'd have to do a lot more thinking before that could become the case), but my confidence in my own view has just taken a serious blow.
I have some position, which I am quite sure is right. I am very sure that my view is well-considered and accounts for almost all possible challenges, and while I'm aware of some potential reasons to be slightly less than absolutely certain of my belief, nevertheless I have put considerable effort into examining my view, considering challenges to it, etc. I am also aware that there exist many specious and nonsensical arguments against this view, and I am reasonably familiar with the broad categories thereof, and have invested effort into carefully considering many (though not all) of them. The "evidentiary crossfire" of all the positive reasons to be confident that my view is right, and the lack of any really compelling reasons to doubt, adds up to a high degree of confidence.
Someone then makes an argument against my view. I have not encountered this specific argument before, but it resembles various other arguments I've seen, and adds basically nothing new to my understanding. I have no immediate reply to make, but hearing this argument doesn't surprise me at all. I already knew that there are many of these bad arguments out there, and having the answers to every single one of them ready to hand simply does not strike me as a good use of my time. I do intend to reply to this one, but not at once. My confidence is not shaken at all.
There exist many bad arguments for various completely wrong and extremely stupid ideas, which I have no interest in spending all my time rebutting. I encounter one such argument for one such idea, to which I have no ready reply. I shrug. It means nothing. Arguments? You can prove anything with arguments.
[Regarding not being able to claim to be unmoved:] One distinction I think it's useful to make is between "you are convinced" and "you think that you should be convinced".
Many people don't make this distinction. It is, I've found, a skill—indeed, one may call it a rationalist skill—to distinguish between "I have heard arguments for X, and I don't know of any rebuttal" (or any number of more or less related epistemic states, like "I've been told X, by someone who I hold in esteem"), and "I actually, for real, think X" (a.k.a., simply, "X"). (Luria's peasants had this skill, though! And it might also be said that this is nothing more than Hanson's "far mode vs. near mode"...)
Scott's hypothetical engineer who thinks "oh, there's a good argument for terrorism; I guess I should become a terrorist" lacks this skill. Many "rationalists", in my experience, also lack it. (This does much to account for the persistent popularity of utilitarianist moral views.)
So, someone makes an argument, which seems to have no obvious flaws; you have no immediate reply. Alright. The one comes to you and suggests that you should therefore be convinced of the argument's conclusion, and should update your own views accordingly.
But are you convinced? Are you, actually, in fact, convinced? Do you actually now believe that your previously expressed position is mistaken? Contrast my scenario 1 with scenario 3: in #1, I am convinced! Actually, genuinely, I now believe differently than I did before! I do not need to ask myself if I should be convinced; I simply am convinced. In #3... not so much.
(One may argue, here, that the presence of bias means that the question "are you actually convinced" is unreliable as a guide to truth. Yes, perhaps, but note that this does nothing to rescue the alternative. If you have biases which prevent you from being actually convinced when a perfectly unbiased reasoner would, in your place, be convinced, then you should of course try to overcome those biases and rectify your epistemic outlook, but it is foolish in the extreme to say "I am biased, therefore I will simply pretend that I am convinced whenever it seems like I should be convinced". That's how you end up on the proverbial train to crazytown!)
What I am advocating is to not pretend that you're convinced. If you're convinced, fine. If you're not, then you're not. Maybe because you're in scenario #3 or #4, maybe because you haven't had a chance to think, maybe who knows why. This is right and proper.
Yea, though I walk through the valley of arguments, I will fear no argument, for I understand that adversarially selected reasoning-chains are memetic hazards.
Said Achmiz, citing Arthur Schopenhauer's The Art of Controversy, argues that debaters encountering an apparently crushing counterargument should not immediately concede:
Achmiz would later describe the behavior of being unwilling to budge to counterarguments as "normatively correct".
I opined that while I agree that it's possible to concede an argument too early and that good outcomes often result from being obstinate in the heat on an argument and reflecting at leisure later, I think describing obstinancy as "normatively correct" is taking it way too far.
In discussions of rationality, I usually take the word normative—of or relating to an authoritative standard—to refer to the authoritative standard of ideal Bayesian reasoning, often in contrast to what humans do. As Schopenhauer observes, the entire concept of adversarial debate is non-normative in this sense! Aumann's agreement theorem is a mathematical expression of Schopenhauer's dictum that "we should not in the least care whether the truth proved to be in favor of the opinion which we had begun by expressing, or of the opinion of our adversary." And yet people do.
Achmiz replied:
Retreating to a less censorious venue in which this discussion about social epistemology would be permitted to continue, I replied:
Another commenter suggested that when faced with an obstinate interlocutor, one should troll them with, "But you know I'm right; you just don't admit it."
Achmiz said:
I said:
Achmiz replied:
I granted that one shouldn't update much at first if the process of evaluating arguments is naturally slow, but elaborated on why I thought the lack of a ready response was informative:
Achmiz replied:
I said:
Achmiz replied:
I clarified:
Achmiz didn't understand:
I articulated a toy model:
Achmiz wasn't buying it:
I confessed:
Achmiz had the last word: