LESSWRONG
LW

833
Wikitags

Tiling Agents

Edited by markov, Mateusz Bagiński, et al. last updated 16th Jul 2024

An agent might have the ability to create similar or slightly better versions of itself. These new agents can in turn create similar / better versions of themselves, and so on in a repeating pattern. This is referred to as an agent tiling itself.

This leads to the question: How can the original agent trust that these recursively generated agents maintain goals that are similar to the original agent's objective?

In a deterministic logical system, assuming that all agents will share the same axioms, "trust" arises from being able to formally prove that the conclusions reached by any subsequently generated agents will be true. The possibility to be able to have this form of trust is influenced by Löb's theorem. The inability to form this trust is called the Löbian obstacle.

See Also: Löbian obstacle, Löbs theorem, Vingean Agents, Vingean Reflection

References :

  • intelligence.org/files/TilingAgents.pdf
Subscribe
Discussion
2
Subscribe
Discussion
2
Posts tagged Tiling Agents
10
88Tiling Agents for Self-Modifying AI (OPFAI #2)
Eliezer Yudkowsky
12y
259
10
37Vingean Reflection: Reliable Reasoning for Self-Improving Agents
So8res
11y
5
10
29Walkthrough of the Tiling Agents for Self-Modifying AI paper
So8res
12y
18
9
14Probabilistic Tiling (Preliminary Attempt)
Ω
Diffractor
7y
Ω
8
9
4Logical Inductor Tiling and Why it's Hard
Ω
Diffractor
7y
Ω
0
6
264Leaving MIRI, Seeking Funding
abramdemski
1y
19
4
62Seeking Collaborators
Ω
abramdemski
1y
Ω
15
3
35The alignment stability problem
Ω
Seth Herd
3y
Ω
15
3
8Paraconsistent Tiling Agents (Very Early Draft)
Ω
IAFF-User-4
11y
Ω
5
3
6Tiling agents with transfinite parametric polymorphism
Squark
12y
11
2
133The Pando Problem: Rethinking AI Individuality
Ω
Jan_Kulveit
8mo
Ω
14
2
66Working through a small tiling result
Ω
James Payor
6mo
Ω
9
2
38Lecture Series on Tiling Agents
abramdemski
10mo
14
2
16Lecture Series on Tiling Agents #2
abramdemski
10mo
0
1
13Rational Effective Utopia & Narrow Way There: Math-Proven Safe Static Multiversal mAX-Intelligence (AXI), Multiversal Alignment, New Ethicophysics... (Aug 11)
ank
9mo
8
Load More (15/16)
Add Posts