The goal of this post is to collect feedback on a new project idea. This work would be a collaboration between the AI Objectives Institute (AOI), MetaGov and other partner institutions.

Project PIs: Bruno Marnette (AOI) and Philipp Zahn (MetaGov, 20squares).
Special thanks to Colleen McKenzie, Matija Franklin, Timothy Telleen-Lawton, Gaia Dempsay, Ping Yee, Justin Stimatze, Cleo Nardo, Tushant Jha, Deger Turan and others for feedback and review.

Motivations

Humans routinely model what the incentives of other humans may be. Historians look at the incentives of powerful individuals to explain why they made specific decisions. Journalists look at the incentives of large companies to uncover collusions and conflicts of interests. Economists use incentive models to understand markets. Lawmakers look into existing incentives before introducing new ones.

There is however a natural limit to how much modeling and analysis a single person or a small team can do and this is where we see an opportunity to leverage LLMs. Instead of relying mostly on human labor to build incentive models, we can now delegate more of the work to machines to reduce cost and increase the scale. There are two main advantages:

LLM-powered incentive models could process larger amounts of information, thus making them more complete, more balanced, and more accurate.
The automation and flexibility of an LLM pipeline allows for rapid iteration and refinement. We can adapt our models by simply adjusting prompts.

Working definitions

We define "incentive" as "factors that may encourage an entity to take a particular action or engage in a specific behavior". The word "factors" can point to different types of motivations, forces, pressures, or rewards and punishments. Depending on the application domain, we may be interested in capturing financial, legal, social, cultural, ideological, or psychological factors.

Research questions

There is already a large body of evidence confirming that LLMs can perform in-depth analysis of documents, as long as they are prompted in the right way and provided with the right context. For example, a MIT paper has illustrated very well how LLMs can be used to produce precise and formal models using a probabilistic programming language. An Anthropic paper has shown how LLMs with large context windows can...

Posts

Wikitag Contributions

Comments