There can be an inherent illogic to human evil. In AI safety, it is important to consider whether a rogue AI agent is capable of committing this type of evil, and if so, what kind of agent might pose a risk of doing so. I draw on work from Corin Katzke and Joseph Carlsmith to explore the power-seeking tendencies of AI, focusing particularly on Carlsmith’s “messy” and “clean” goal directedness model, to conceptualise the nature in which an AI agent may acquire power-seeking characteristics. I then look at instances of catastrophic power-seeking carried out by humans: genocides, asserting that genocide is a product of “messy” goal directed-ness, and following this I question if the nature of this power-seeking could be similar to that which an AI agent might develop. My overall argument is that deficiencies in the cognitive architecture of AI agents may act as a gateway for such agents to commit ‘human-like’ evil.

Theories of agency

Corin Katzke conceptualises transformative AI in a manner which grapples with the fundamental questions of what it means to be an agent, how AI agents may develop power-seeking tendencies, and how power-seeking may become an intrinsic, overarching goal for a superintelligent AI agent. In doing so, he draws on Joseph Carlsmith’s framework for categorising the goal directed-ness of AI agents on a scale running from “messy” to “clean”. Using this model, Katzke suggests that the “cleaner” an agent is, the more likely that it will develop power-seeking attributes.

To better understand this distinction between “messy” and “clean” it is necessary to explain the role of heuristics in cognitive function. Heuristics are cognitive shortcuts which learn from patterns and regularities in environments in order to make inferences about new aspects of an environment. This allows an agent to simplify interpretations of their environment and effectively make decisions within it. For example, humans tend to be good at picking up on body language, and interpreting body language to make judgements about the mood or temperament of another person. These judgements are not calculated by analysing every single aspect of a person’s appearance, but rather by recognising certain gestures and facial expressions which hold similarities to previously witnessed gestures in past social situations. It is through reference to patterns or repetit...

Posts

Wikitag Contributions

Comments