Embedded Agency

Daniel_Eth (+8/-7)
Vladimir_Nesov (-630)
Noosphere89 (+640/-10)
Noosphere89 (+314/-43)
Noosphere89 (+10/-10)
Ruby
Ben Pace
Ben Pace (+707)

Embedded Agency is the problem that an understanding of the theory of rational agents must account for the fact that the agents we create (and we ourselves) are inside the world or universe we are trying to affect, and not separated from it. This is in contrast with much current basic theory of AI or Rationality (such as Solomonoff induction or Bayesianism) which implicitly supposes a separation between the agent and the-things-the-agent-has-beliefs about. In other words, agents in this universe do not have Cartesian or dualistic boundaries like much of philosophy thinks,assumes, and are instead reductionist, that is agents are made up of non-agent parts like bits and atoms.

A second problem that Embedded Agency tries to solve is how bounded reasoners in both computation, data and time reason in a world bigger than us, without the aid of omniscience or infinite time. Bayesianism, probably the biggest improvement in rationality before Logical Induction was invented by Scott Garrabrant and Abram Demski, relies on logical omniscience to do well, but real agents can't be omniscient.. Thus the Embedded Agency program was started under the leadership of Abram Demski and Scott Garrabrant to solve the problem of how to reason in an environment much bigger than you without omniscience or infinite time.

A second problem that Embedded Agency tries to solve is how bounded reasoners in both computation, data and time reason in a world bigger than us, without the aid of omniscience or infinite time. Bayesianism, probably the biggest improvement in rationality before Logical Induction was invented by Scott Garrabrant and Abram Demski, relies on logical omniscience to do well, but real agents can't be omniscient.. Thus the Embedded Agency program was started under the leadership of Abram Demski and Scott Garrabrant to solve the problem of how to reason in an environment much bigger than you without omniscience or infinite time.

Embedded Agency is not a fully formalisedformalized research agenda, but Scott Garrabrant and Abram Demski have written the canonical explanation of the idea in their sequence Embedded Agency. This points to many of the core confusions we have about rational agency and attempts to tie them into a single picture.

Embedded Agency is an intuitive notionthe problem that an understanding of the theory of rational agents must account for the fact that the agents we create (and we ourselves) are parts ofinside the world,world or universe we are trying to affect, and not separated from it. This is in contrast with much current basic theory of AI or Rationality (such as Solomonoff induction)induction or Bayesianism) which implicitly supposes a separation between the agent and the-things-the-agent-has-beliefs about. In other words, agents in this universe do not have Cartesian or dualistic boundaries like much of philosophy thinks, and are instead reductionist, that is agents are made up of non-agent parts like bits and atoms.

Embedded Agency is an intuitive notion that an understanding of the theory of rational agents must account for the fact that the agents we create (and we ourselves) are parts of the world, and not separated from it. This is in contrast with much current basic theory of AI (such as solomonoffSolomonoff induction) which implicitly supposes a separation between the agent and the-things-the-agent-has-beliefs about.

Embedded Agency is an intuitive notion that an understanding of the theory of rational agents must account for the fact that the agents we create (and we ourselves) are parts of the world, and not separated from it. This is in contrast with much current basic theory of AI (such as solomonoff induction) which implicitly supposes a separation between the agent and the-things-the-agent-has-beliefs about.

Embedded Agency is not a fully formalised research agenda, but Scott Garrabrant and Abram Demski have written the canonical explanation of the idea in their sequence Embedded Agency. This points to many of the core confusions we have about rational agency and attempts to tie them into a single picture.