I don’t agree that the epistemic standard you claim this violates is actually an epistemic standard. There is no rule that I can’t point out my conversational partner may be falling prey to a cognitive bias. In fact, the authors are being radically honest by explicitly stating they expect motivated disagreement. Then they explore more detailed arguments for their position, in order to counter that motivated disagreement.
The question in the title of this post is apparently about something other than and not very closely connected to the actual point of the post.
Thank you. Your representation of my argument has all the flaws you find, but look again to make sure it's the right representation of my argument.
Ad hom attacks may be radically honest in the way that any other violence to sincere engagement is, but the radical honesty to admire admits one's own potential for motivated reasoning in the same breath. I don't see that humility from the authors.
I'm not concerned that they are primed to expect motivated disagreement, but that they are primed to expect only motivated disagreement; that no one who disagrees is capable of engaging sincerely; that there is no rational disagreement. It's OK to think that. That's what it means to be axiomatic. The rule is you've got to say so. And I'm certainly not alone in noting that the authors are very dismissive of failures to accept their axiom-in-argument's-clothing.
It's entirely possible I haven't read enough outside the book, refs are welcome. If I am stuck in the labyrinth of emotional resistance to such brave truths, I don't have to be reminded of that possibility. What will give me a chance at peeling the scales from my eyes is 1. engagement with the counterarguments in their strongest form assuming equally rational interlocutors, or 2. an invitation to just take that particular point as given and see where it leads.
Sometimes you can’t assume equally rational interlocutors, because your interlocutors are not equally rational, and that would be a false assumption.
That’s different from failing to address the arguments in their strongest form. I think you are equating these two separate critiques.
What is the strongest argument, in your view, that was not addressed?
> That’s different from failing to address the arguments in their strongest form. I think you are equating these two separate critiques.
I think not. You'll find elsewhere on the site that the variations of Steelmanning, the Principle of Charity, and Assume Good Faith are all cited adjacently to each other. The quite reasonable and unremarkable epistemic standard I'm setting is the neighborhood of these ideas, and I'm surprised to see so little interest in it.
> Sometimes you can’t assume equally rational interlocutors, because your interlocutors are not equally rational, and that would be a false assumption.
And there is no interlocutor. It's a book, with a wide audience that won't talk back. A writer lacks full dossiers on their complete readership, and so usefully/incorrectly assumes an audience capable and invested in understanding. Otherwise why write? Having assembled such an audience to tell them there's no reasonable disagreement with a key premise of the book just feels like a missed opportunity, and out of the spirit of like the whole enterprise.
> What is the strongest argument, in your view, that was not addressed?
In the interest of staying scoped, I'll call it a subject for another thread post. As a summarizing statement, I'm coming away from this exchange at least as concerned that I'm engaging in a community that has rationalized itself into thinking that ad hominum attacks and dismissive assumptions of bad faith are how we arrive at shared truth. I would of course love to be wrong. Thank you for engaging to this point.
I think not. You'll find elsewhere on the site that the variations of Steelmanning, the Principle of Charity, and Assume Good Faith are all cited adjacently to each other. The quite reasonable and unremarkable epistemic standard I'm setting is the neighborhood of these ideas, and I'm surprised to see so little interest in it.
But they are not the same thing, and none of this proves that you can't point out a possible cognitive bias.
And there is no interlocutor. It's a book, with a wide audience that won't talk back. A writer lacks full dossiers on their complete readership, and so usefully/incorrectly assumes an audience capable and invested in understanding. Otherwise why write? Having assembled such an audience to tell them there's no reasonable disagreement with a key premise of the book just feels like a missed opportunity, and out of the spirit of like the whole enterprise.
The authors do not claim any specific reader is subject to motivated reasoning. They simply state that they believe the argument made so far is sufficient, before moving on to further arguments, for those who may not be convinced.
You argue that a book must assume an audience invested in understanding etc. I do not think the authors are treating the reader as "not invested in understanding," simply fallible. And readers are in fact fallible, and there is no need to assume otherwise in order for a book to be worth reading. It's very useful to model the reader's state of mind (and this is necessary to write good educational materials). I think your point would prove too much, if true.
With that said, I ALSO don't really read this statement as necessarily an accusation that the reader may be engaged in motivated reasoning. It's more like "okay if you're with us you should be convinced now, but here are some common counterarguments and our responses" which I think is still potentially of interest if you are convinced. In other words, I do not feel patronized by this, I see it as a useful sign-post that the most load-bearing part of the argument has passed. The rest are useful illustrative details and potentially can be borrowed for discussions with others who may not be convinced by the load-bearing part.
In the interest of staying scoped, I'll call it a subject for another thread post. As a summarizing statement, I'm coming away from this exchange at least as concerned that I'm engaging in a community that has rationalized itself into thinking that ad hominum attacks and dismissive assumptions of bad faith are how we arrive at shared truth. I would of course love to be wrong. Thank you for engaging to this point.
This is not out of scope, but is actually central to the discussion. There's no point in making this meta-level critique without an object-level critique to back it. (See also https://www.greaterwrong.com/posts/tMhEv28KJYWsu6Wdo/kensh/comment/hYiyo8JbgXtT5wcZx)
I see that you are concerned about my epistemics. The feeling is mutual. For example, where did this criticism on ad hominum attacks appear from? Are you equating the discussion of cognitive biases with ad hominem attacks? And worse, how exactly does this one discussion with me implicate the epistemics of the entire lesswrong community? That evidence is obviously incredibly weak, for better or worse. But I mostly prefer to stay on the object level here, and have tried to avoid straying too far in this direction.
I would argue that the statement "Making a future full of flourishing people is not the best, most efficient way to fulfill strange alien purposes" is nearly tautological for sufficiently established contextual values of "strange alien purposes". What is less clear is whether any of those alien purposes could still be compatible with human flourishing, despite not being maximally efficient. The book and supplementary material don't argue that they are incompatible, but rather that human flourishing is a narrow, tricky target that we're super unlikely to hit without much better understanding and control than our current trajectory.
The relevant epistemic standard is "if you want to find out your opponent's objections,don't guess, ask them". Sceptics actually don't accept the premise that an AI trained on a human generated corpus will be weird and alien.
For me the 0th rule of rational dialogue is common assumption of commitment to the finding the truth together. That's a belief in your audience's sincere interest in the truth and capacity for it.
In their book, Nate and Eliazer seem to fail that standard with rhetorical sappers like "bitter pill":
"""
Making a future full of flourishing people is not the best, most efficient way to fulfill strange alien purposes. So it wouldn't happen to do that, any more than we'd happen to ensure that our dwellings always contained a prime number of stones. In a sense, that's all there is to it. We could end the chapter here. But over decades of experience, we have found that this bitter pill is often hard for people to swallow.
"""
If you are dismissing disagreement or lack of understanding as emotional avoidance, you are not engaging with your audience as if they're able and willing to understand you. How does that serve a rationalist cause?
Not that there's anything wrong with the literary form of the political statement, but I read someone as arguing in bad faith if they're passing their argument off as anything more.
A responsible way of handling would be to say
"We will end the chapter here. An assumption of this work is that AI indifference is fundamentally opposed to wellbeing. Since we believe that that is true on face, we don't argue it in this work, and you will have to judge our conclusions on the assumption that it holds in order to be experienced as engaging with us capably and sincerely."
To say anything else is a failure to acknowledge the existence of legitimate disagreement, and we're into theology.
It's been a while since I posted on this forum, but I wanted to check the water.