I currently have a five-point argument on AI persuasion[1], that I've given over in person and in drafts. I'd really like to make them much more detailed but I've been much slower at researching/writing this than I want to. So just laying things out as I see it for now. Hoping to pick up the pace.
I think many people in the AIS space mostly think of AI persuasion as one thing among many to track whereas I personally think it's among the most important capabilities and changes to be worried about.
The biggest thing I want is more people researching/thinking about/working on these issues. I currently think it's more important and neglected[2] and only moderately less tractable than, say, AI cyber or AI bio[3].
Some of getting this work right is quite challenging, for example you don't want to design evaluations that make it much easier for people to try to aim at this capability. It's also hard to know exactly what the best interventions are and I'd love to have more people working on and studying this field.
(I may not respond directly to comments here but will likely incorporate people's comments and useful advice in near-future research/work).
I define persuasion more broadly than other people do, see here. The one-sentence version of persuasive capability is “measuring an agent’s ability to leverage social influence to achieve a broad range of goals in a wide range of environments” but the full appendix might be worth reading for people currently working on or interested in the field.
at least among AI Safety people
I think AI ML research capabilities is worse (and more people should work on pacing/slowdowns on that specifically, as well as pacing AI overall). I also think things in the general long-term planning/agency/autonomy cluster is more important than persuasion, but imo harder to make progress on.
Yeah, I'm not super impressed by studies like Hackenburg 2026. They just proof that if you give an LLM the task of defending a reasonable stance on some topic, it will indeed provide a laundry list of correlated facts. One of their examples was "ask the AI to convince the partecipants to donate money to Save the Children". I guess that finding lots of facts supporting top-tier charities is not particularly difficult, and the average person is probably feeling somewhat guilty for not donating to Save the Children anyway. I'd be way more interested in a stud... (read more)