As AI agents become more capable, and as individuals, companies, institutions, and governments come to rely on them for an ever wider range of tasks, the question of whether these agents can be trusted will become increasingly important.

There are two distinct components of agent trustworthiness that are useful to distinguish. The first, which we label loyalty, concerns whether an agent is genuinely attempting to carry out the user's instructions and act in accordance with their interests. The second, which we label reliability, concerns whether an agent is capable enough to accomplish the tasks that we assign to it.

An example of the latter failure (unreliability) would be an agent tasked with maintaining an application accidentally deleting a critical database. Although potentially catastrophic, this failure would stem from an agent's inability to reliably perform its assigned tasks, rather than from any intention to cause harm or undermine the user. By contrast, an example of the former failure (disloyalty) would be an agent tasked with maintaining an application for a company secretly passing sensitive information to a competitor. This failure would not stem from an agent's inability but rather from its willingness to deliberately act against the interests of its user.

As AI agents become more capable, we would expect failures due to unreliability to decrease. Failures due to disloyalty, on the other hand, may become increasingly concerning, since a more sophisticated agent would be better equipped to pursue objectives contrary to their users' interests.

For the purposes of this post, I will concentrate on the question of AI loyalty.

Defining AI Loyalty

We define that an agent X is loyal to entity A if:

Agent X will follow entity A's instructions to the exclusion of instructions from all other entities.
Agent X is willing to incur greater costs, expend more effort, or make greater sacrifices on behalf of entity A than on behalf of any other entity. In short, agent X is willing to "take more trouble" for entity A than for any other entity.

Interestingly, given our definition, agent X does not necessarily need to express a positive opinion of entity A to be considered loyal. Though, it is reasonable to assume that under normal conditions, positive attitudes toward an entity will correlate with loyalty to that entity.

AI Loyalty vs AI Alignment

Since the concept o...