TL;DR: In terms of the potential risks and harms that can come from powerful AI models, hyper-persuasion of individuals is unlikely to be a serious threat at this point in time. I wouldn’t consider this threat path to be very easy for a misaligned AI or maliciously wielded AI to...
This post examines AI alignment through the lens of systems thinking and safety engineering. We aim to identify structural mechanisms that can maintain alignment in complex sociotechnical systems, systems where AIs interact with multiple human operators and stakeholders.[1] One conception of AI misalignment is a control problem where the behavior...
In 1984, Charles Perrow wrote the book Normal Accidents: Living with High-Risk Technologies. The book is an examination of the causes of accidents and disasters in highly complex, technological systems. In our modern time it can be helpful to review the lessons that Perrow set forth, as there may be...