Review
TL;DR: A series of valid bounded arguments all arguing the same proposition can only provide as much evidence as that proposition at best, so even if it looks like they're piling up in favor of a higher point, they're only as good as the most central of them.
Epistemic status: Armchair epistemology. Something I noticed + anecdotal evidence.
During the LWCW last WE, I played an activity called Steelmanning the devil, where you had to pick a position you disagreed strongly with and steelman it. Participants were scored on their number of valid arguments and fallacies: 1 point for a valid argument, 0.5 point for an unconvincing argument, -1 point for a fallacy (with a lot of room for human judgment so score reflects how convincing the participants were).
I won using arguments which did not prove my point, which I will call piling bounded arguments, and I want to point them out because I think it's easy to be confused online because of them, and they're apparently not common knowledge enough that I wasn't called out during a rationalist competition about arguing.[1]
The activity went like thus: you named an opinion A you disagreed with (if possible, on a tribal level); you were paired with someone who'd attack A and you'd do your best to steelman it.
You disagree with A. You are informed on the subject, you are confident in your opinion because you've looked at all arguments and found them, on balance, to tend towards A and that arguments for A are generally subtly (or non-subtly) flawed.
This does not mean that there is no valid argument for A. They might just be too weak or too few compared to the arguments against.
Suppose you know one good argument for A (call it P). You share it. Your partner says
"But what about X? That would make P invalid, so you don't have evidence in favor of A."
"Actually, Q=>X. This is valid, central, and final. Nothing left to argue about X, it was just plain wrong."
"Right. What about Y? Idem."
"Well, R=>Y."
"What about Z? Idem."
"Nice one, but it's a non-central counter-argument; the central part of P is still valid."
And so on; after half an hour of arguing like that, your partner might be quite impressed that you can defend all counter-arguments and figure out that in the end, A must be true (or at least that there is a lot of strong evidence for it). But you only ever discussed the validity of P, so your partner shouldn't be anymore impressed than if they'd just accepted P outright (or maybe a little more to account for the now unlikelier possibility the argument does not apply).
Now let's keep track of probabilities. Your partner's prior belief is 20% or 1:4. P is 2:1 evidence in favor of A, so they update towards 2:4 -> 33%. Each time they counter-argue, they regress towards 20%; each time you counter-counter-argue, they move up towards 33%. At no point should they go above 33%.
But people don't actually work like that; like the competition, they'll do something more akin to keeping track of the good and bad arguments, and distill a total score from them. It sure looks like there are a lot of arguments in favor of A and that all the arguments against A have been thoroughly debunked!
The value of the bounded arguments depends a lot on the value of the central argument they're supporting. If the central argument is a crux, then piling arguments for it is fine.
If you notice piling bounded arguments, consider: if the central argument was so true as to be beyond debating, what would it change? Can you replace the pile by the central argument and continue the debate?
(If it's a friendly debate, where you're both arguing as best you can to find the truth together, then it's often beneficial to accept an argument and move on rather than dig into the deepest roots of validity. It's hard to notice how many independent arguments are made in favor of a point otherwise.)
If you notice piling bounded arguments, but they're all invalid, motivated, etc. then they only detract from their central argument. It should not make you go below what you'd believe if the central argument of the pile had been advanced without justification (except insofar as you expected better justification from your partner).
There is probably a way in which piling bounded arguments can be used to fool people; like other dark arts, they'll be especially infuriating to your opponents and effective on the audience; since all your counter-counter-arguments are valid, it's also easy to think that you're arguing in good faith while giving them.
The way I go about it, during a debate-like discussion, is to build a dependency-tree of arguments, all weighted by how relevant they are to the parent nodes, which I can prune when one is found invalid.[2]
It beats trying to iteratively memory-lessly update a belief as arguments come in because a proposition often moves around as it is being argued, to better fit the arguments: it is simpler to change the conclusion so it is relevant than to keep track of exactly how much relevant the argument is.[3]
A problem with it is that after the debate, you have to keep only the conclusion because it's too expensive to keep track of the arguments. It's good enough, though I am not aware if it is a theoretically sound approach.
Epistemic status: random thoughts, mostly an excuse to share classsic articles.
I don't often encounter situations where I'm like "Hey, these are bounded-value arguments! The discussion would be fine without those, but as is, it needs to be sanitized by calling out the bounded-value arguments!"
They're just not that common, and not that useful as dark arts compared to the usual methods.
Nonetheless, they can be used to facilitate confirmation bias among rationalists (when you have a motivation to adopt a strategy for evaluating truth that conveniently falls prey to bounded value arguments, without compromising on argument validity).
When I try to categorize bounded-value arguments in a taxonomy of fallacies and biases, they sound like a variation of the non-central fallacy, because they are a method for obfuscating the relevance of the arguments to the central point of discussion (though they hinge on how much relevant rather than whether relevant).
They also seem like the big brother of double-counting evidence? The double-counter beats all counter-arguments with a single invalid one, whereas the bounded value arguer beats each counter-argument with a valid one, but still gets it wrong.
No social media example comes to mind. Please provide them in commentary or I guess it'll mean piling bounded arguments aren't that important to know about.
I had a lot of fun in Steelmanning the devil but it didn't seem like a particularly effective way to improve epistemology (especially since we didn't share tips afterwards). It didn't seem like a particularly effective way to evaluate epistemological skill either.
I don't recommend this specific epistemology-building activity over others, but I do recommend them in general. Thanks to Sam for organizing this one!
It wasn't like, a super-competitive competition which only lsusrs on steroids would attend. Participants fell prey to the fallacies we all know and love.
I did expect better though... talk about in-group bias!
This is not optimal and please tell me if you know a more principled method that works in practice.
Also, disclaimer: that's not how it works in practice. In practice, you know the various methods and sorta apply them and it sorta works, but most of the cognitive work is unconscious and illegible; I feel like I have a weighted tree in my head, but I'm not able to list all arguments and how much they support each other.
Alice: "The sky is blue."
Bob: "I see the sky, there are clouds, it's grey."
Alice: "The sky is quite grey right now, but blue is its default color; when I said 'The sky is blue', I meant it, like, as a general truth."
Bob: "The sky is generally blue, because we generally observe that it is blue. The fact that I currently observe that it is grey is (weak) evidence that it is generally not blue (and more particularly generally grey)."
Alice: "Yes. But we do agree that as a general truth, the sky is blue?"
Bob: "Somewhat. It's a standard example for argumentation; but as we've shown, it's not a good example of something generally true."
Here, Alice and Bob can be equivalently understood to argue on the meaning of the statement "the sky is blue" or on its validity.
One might take away that the sky is indeed blue, for a certain meaning of the statement, or one might take away that there is good evidence the sky is blue in certain situations, though there are good counter-arguments in other situations.