LESSWRONG
LW

2800
Wikitags
You are viewing version 1.4.0 of this page. Click here to view the latest version.

Fai-Complete

Edited by Alex_Altair, Costanza R, et al. last updated 31st Jan 2013
You are viewing revision 1.4.0, last edited by Costanza R

A problem is Friendly AI-complete if solving it is equivalent to creating Friendly AI. Friendliness theory is one proposal among many for ushering in safe Artificial General Intelligence technology. The Singularity Institute argues that any safe superintelligence architecture is FAI-complete. For example,

  • Oracle AI
  • Tool AI
  • Nanny AI

In "Dreams of Friendliness" Eliezer Yudkowsky argues that if you have an Oracle AI, then you can ask it, "What should I do?" If it can answer this question correctly, then it is FAI-complete.

Similarly, if you have a tool AI, it must make extremely complex decisions about how many resources it can use, how to display answers in human understandable yet accurate form, et cetera. The many ways in which it could choose catastrophically require it to be FAI-complete. Note that this does not imply that an agent-like, fully free FAI is easier to create than any of the other proposals.

Goertzel proposed a "Nanny AI"(Should humanity build a global AI nanny to delay the singularity until it’s better understood?) with moderate superhuman intelligence, able to forestall Singularity eternally, or to delay it. However, it has been noticed by Luke Muehlhauser and Anna Salamon (Intelligence Explosion: Evidence and Import) that a Nanny AI is FAI-complete. In fact, building it would require to solve the problems required to build Friendly Artificial Intelligence.

Blog posts

  • Dreams of Friendliness
  • Reply to Holden on 'Tool AI'
  • A Taxonomy of Oracle AIs

References

  • Should humanity build a global AI nanny to delay the singularity until it’s better understood? by Ben Goertzel
  • Intelligence Explosion: Evidence and Import by Luke Muehlhauser and Anna Salamon

See also

  • FAI
  • AI-complete
Discussion
1
Discussion
1