A logical decision theory recommends that you choose as if deciding the output of your decision algorithm. The main difficulty in formulating a logical decision theory is how to define statements like: "If my algorithm outputs this, the result will be that". These look like counterfactual implications, but counterfactuals describe...
MIRI's proof-based prisoner's dilemma tournament defined agents encoded as formulas of Peano arithmetic (PA) with one free variable. means the agent cooperates in a match against the agent , and is constructed by plugging the Gödel number of the formula defining into the formula defining . The simplest interesting agent...
Recently I’ve been thinking a lot about a certain model of a rational agent: a proof-based agent which is triggered to act when it finds certain proofs in Peano arithmetic (PA). Back when MIRI had an agent foundations team, they found they could derive what these agents would do using...
In his original paper on what we now call the "many-worlds" interpretation, Everett motivated it with quantum cosmology, since there's nowhere outside the universe for a Copenhagen-style observer to stand. Eliezer Yudkowsky said something similar to motivate timeless decision theory: > I hold it a virtue of any decision theory...
The context is MIRI's twist on Axelrod's Prisoner's Dilemma tournament. Axelrod's competitors were programs, facing each other in an iterated Prisoner's Dilemma. MIRI's tournament is a one-shot Prisoner's Dilemma, but the programs get to read their opponent's code. Or, rather, a description of the behavior of the code in Gödel-Löb...
The sunlight hitting even a small portion of the United States has enough energy to power the whole country—that simple calculation was the subject of the last post. All this shows on its own is that running the country on solar power is physically possible. With photovoltaics it may be...