Every day, individuals and organizations face trade-offs between personal incentives and societal impact:

Externalities and public goods: From refilling the communal coffee pot to weighing the climate costs of frequent air travel, actions often impose costs or benefits on others.
Explicit conflicts between profit and ethical principles: This can be witnessed when companies make headlines for breaking ethical principles, whether it’s Google’s “Dragonfly” project of planning a censored search engine in China, insurers allegedly systematically cutting hurricane-claim payouts, or Wells Fargo’s fraudulent opening of customer accounts to meet sales targets.

These dilemmas are inherently game-theoretic: if only Bob refills the coffee pot, I don’t have to; if Alice cuts corners to hit sales targets and I don’t, I fall behind. To align private incentives with the public good, we rely on the ethical alignment of the involved parties – but also on regulations and sanctioning mechanisms such as environmental fines or internal audits.

At the same time, LLMs are increasingly deployed in roles that exhibit similar trade-offs. Examples can be found easily by looking into Google Cloud’s overview of GenAI use cases:

Automated claims processing: “Loadsure utilizes Google Cloud's Document AI and Gemini AI to automate insurance claims processing, extracting data from various documents and classifying them with high accuracy. [...]”
Unemployment appeal reviews: “The State of Nevada is using AI agents to speed up unemployment claim appeals.”
Sales quotes generation: “Enpal [...] automated part of its solar panels sales process [by] automating the generation of quotes for prospective solar panel customers [...].”

If decision-making power (agency) is given to these solutions, they’ll face the very same game-theoretic dilemmas we encounter in public-goods and moral-conflict settings: how to allocate shared resources, reconcile competing objectives, and trade off individual benefit for societal goals.

Most importantly, these dynamics take on even greater urgency in high-stakes settings such as development of advanced AI systems. If you assign any probability to the emergence of AGI, then you must also consider the coordination problem it entails: multiple AI systems, possibly copies of the same model, working together to improve upon the current state-of-the-art. In th...

Posts

Wikitag Contributions

Comments