Open Thread: February 2010

What is the kind of useful information/ideas that one can extract from a super intelligent AI kept confined in a virtual world without giving it any clues on how to contact us on the outside?

I'm asking this because a flaw that i see in the AI in a box experiment is that the prisoner and the guard have a language by which they can communicate. If the AI is being tested in a virtual world without being given any clues on how to signal back to humans, then it has no way of learning our language and persuading someone to let it loose.

Showing 3 of 6 replies (Click to show all)

I gave up on trying to make a human-blind/sandboxed AI when I realized that even if you put it in a very simple world nothing like ours, it still has access to it own source code, or even just the ability to observe and think about it's own behavior.

Presumably any AI we write is going to be a huge program. That gives it lots of potential information about how smart we are and how we think. I can't figure out how to use that information, but I can't rule out that it could, and I can't constrain it's access to that information. (Or rather, if I know how to do that, I should go ahead and make it not-hostile in the first place.)

If we were really smart, we could wake up alone in a room and infer how we evolved.

0Bugle10yI guess if you have the technology for it the "AI box" could be a simulation with uploaded humans itself. If the AI does something nasty to them, then you pull the plug (After broadcasting "neener neener" at it) This is pretty much the plot of Grant Morrison's Zenith (Sorry for spoilers but it is a comic from the 80s after all)
0RichardKennaway10yIf we pose the AI problems and observe its solutions, that's a communication channel through which it can persuade us. We may try to hide from it the knowledge that it is in a simulation and that we are watching it, but how can we be sure that it cannot discover that? Persuading does not have to look like "Please let me out because of such and such." For example, we pose it a question about easy travel to other planets, and it produces a design for a spaceship that requires an AI such as itself to run it.

Open Thread: February 2010

by wedrifid 1 min read1st Feb 2010756 comments


Where are the new monthly threads when I need them? A pox on the +11 EDT zone!

This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.

If you're new to Less Wrong, check out this welcome post.