Introduction

This blog post serves as an introduction to Evidential Decision Theory (EDT), designed for readers with no prior background. While much has been written about EDT’s application to well-known problems like the Smoking Lesion problem, and Newcomb-like problems, we are unable to find a clear and formal introduction. As such, this post aims to fill that gap.

Our motivation for writing this post stems from the work of Nate Soares and Benja Fallenstein in their papers, Towards Idealized Decision Theory (2015) [8] and Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda (2015) [9]. These works suggest the need for the study of decision theory in aligning smarter-than-human agents with human interests and address the limitations of existing formulations, such as EDT and Causal Decision Theory (CDT) in defining idealized decision procedures.

This post does not take a stance on the need for an idealized decision procedure or the tractability of one. Rather, we hope it provides a clear introduction to EDT and perhaps encourages readers to explore these topics further.

Decision Theory

Decision theory is a rich interdisciplinary field in which philosophers, psychologists, economists, computer scientists, statisticians and mathematicians aptly study decision. While decision theorists disagree on a lot, they do share some foundational distinctions. An important distinction is between normative and descriptive decision theory.
Descriptive decision theory concerns how people actually make decisions.
Normative decision theory, on the other hand, concerns how agents ought to make decisions. We will focus on the latter.

EDT is a branch of normative decision theory first introduced by Richard C. Jeffrey in The Logic of Decision (1965) [4]. In this work, Jeffrey argues that an agent should evaluate actions based on their evidential implications—specifically, one should choose the action with the highest news value. In the following sections, we will unpack exactly what this means, drawing on both Jeffrey’s work and Ahmed’s modern analysis in Evidential Decision Theory (2021) [1].

Decision Problems

To take a step back, let's formalize what we mean by an agent making a decision.
For the remainder of this report, we will primarily follow Jeffery's formalization of decision problems.^[1] In this framework, an agent is a deci...

Posts

Wikitag Contributions

Comments