This is a special post for quick takes by Linch. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I'm a researcher at Forethought and my current primary focus is on risks from AI persuasion
For scoping purposes I'm most interested in the possibility that AIs become meaningfully superhuman at persuasion relatively early (before or in the early stages of the intelligence explosion). My reasoning is that if superhuman persuasion arrives fairly late, we'd be better able to handle these issues post-ASI, or alternatively we'd already be screwed. And my reason for caring primarily about near-future superhuman persuasion rather than late-2025/early-2026 level persuasion is that I think it's both much more neglected and probably a much bigger deal if true.
I think we have an ontology mismatch. I'm not that interested in arguing about AI-in-a-box like stuff because I don't expect that to be a realistic hypothetical. The affordances we currently give AIs, including in internally deployed models, are already much higher than that.
2StanislavKrym
GPT-4o and some other AIs have already demonstrated the propensity to make delirious claims and have the user believe them.
How similar is this to superpersuasion on which you work?