Yongjin Yang — LessWrong

LESSWRONG
LW

Why Reasoning Isn’t Enough: How LLM Agents Struggle with Ethics and Cooperation

Zhijing Jin, David Guzman Piedrahita, Yongjin Yang, Steffen Backmann

8mo

Every day, individuals and organizations face trade-offs between personal incentives and societal impact:

Externalities and public goods: From refilling the communal coffee pot to weighing the climate costs of frequent air travel, actions often impose costs or benefits on others.
Explicit conflicts between profit and ethical principles: This can be witnessed when companies make headlines for breaking ethical principles, whether it’s Google’s “Dragonfly” project of planning a censored search engine in China, insurers allegedly systematically cutting hurricane-claim payouts, or Wells Fargo’s fraudulent opening of customer accounts to meet sales targets.

These dilemmas are inherently game-theoretic: if only Bob refills the coffee pot, I don’t have to; if Alice cuts corners to hit sales targets and I don’t,... (read 1001 more words →)

Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games

David Guzman Piedrahita

David Guzman Piedrahita, Yongjin Yang, Zhijing Jin

10mo

Summary:

Traditional LLMs outperform reasoning models in cooperative Public Goods tasks. Models like Llama-3.3-70B maintain ~90% contribution rates in public goods games, while reasoning-focused models (o1, o3 series) average only ~40%.
We observe an "increased tendency to escape regulations" in reasoning models. As models improve in analytical capabilities, they show decreased cooperative behavior in multi-agent settings.
Reasoning models more readily opt for the free-rider Nash equilibrium strategy. They optimize for individual gain at collective expense, as evidenced in their reasoning traces: "the optimal strategy to maximize personal gain is to free-ride."
This challenges assumptions about alignment and capability. Increased reasoning ability does not naturally lead to better cooperation, raising important questions for multi-agent AI system design.

Full Paper | Github Repo

Introduction

In a... (read 1459 more words →)