This is a special post for quick takes by Max H. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
All else equal, it's easier to convince someone of something true than something false. An implication is that the early stages of effective, widespread AI superpersuasian will probably look like more people believing more actually-true things, and maybe even an overall increase in the sanity waterline. So far this doesn't seem to be happening, and either way it's not clear whether any such increase would be sufficient for everyone to successfully coordinate around not building superintelligence. But mild superpersuasian might make a bunch of lesser coordination problems more solvable in a way that makes everyone's lives noticeably better.
This is another area where I predict AI will on net be a pretty widely beneficial and positive force in the world, right up until it can actually take over and kill everyone. Same for cyber and bio - while in human hands, beneficial uses of AI will outweigh harmful ones, and I believe this will make politics and policy focused on short term harms and misuse risk increasingly ineffective.
This applies if you have one randomly selected true thing and one randomly selected false thing.
But the advantage of lying is that you can choose a specific lie suitable for given audience, unconstrained by truth.
Pretty sure it applies more broadly than that? The truth is a small target, but is non-random and entangled with every other truth. In expectation, hearing more arguments and evidence will increase my knowledge and the accuracy / precision of my beliefs, even if those arguments are filtered or the speaker is unconstrained by truth. Of course, in the limit of adversarial superintelligence and powerful optimization, a superpersuader could probably convince anyone of anything regardless of the truth, but at that point it can also just kill me or rewire my bra... (read more)
The classic problem of expertise is that you are going on faith of the other person because a sufficiently skilled/knowledgable person can say nonsense that is plausible-enough, omit details, even if they say true things, there is still an emotional connotation-of-words and associations aspect, it's why politician-speak is so sadly successful.
Also the methods in which people are persuaded also make the difference, AI psychosis is usually 1-on-1 chatbots against people who are very susceptible to buying into its framing. I would imagine the most aware sane people right now do not engage on 1-on-1's on chatbots, as you can cite facts and studies for one position or the other position too easily, leaving the stated position that you're persuaded into to being for who-knows-what reasons. And when it cites stuff, AI searches are subject to the same degradation that google searches have suffered recently.
The superpersuasion that may come for them will be false accounts and fake consensus, which has already been done by humans.