[SEQ RERUN] The Design Space of Minds in General

My primary moral is to resist the temptation to generalize over all of mind design space

If we focus on the bounded subspace of mind design space which contains all those minds whose makeup can be specified in a trillion bits or less, then every universal generalization that you make has two to the trillionth power chances to be falsified.

That's what I think every time someone brings up the idea of tortured sims. What are the odds of it happening?

My primary moral is to resist the temptation to generalize over all of mind design space

That's what I think every time someone brings up the idea of tortured sims. What are the odds of it happening?

The odds of that happening are almost entirely unrelated to the proportion of mind-space such torture machines make up. That kind of 'torture AI' is something that would be created - either chosen out of mind space deliberately or chosen by making a comparatively tiny error when aiming for a Friendly AI. It isn't the sort of thing that is just randomly selected.

2DanielLC7yI'd say somewhere in the vicinity of 100% [http://creatures.wikia.com/wiki/Norn_torture]. I'm just hoping euphoric sims will be more common.
1[anonymous]7yDo you have a particular formal specification of a tortured mind in mind?

[SEQ RERUN] The Design Space of Minds in General

by MinibearRex 1 min read14th Jun 201218 comments

2


Today's post, The Design Space of Minds-In-General was originally published on 25 June 2008. A summary (taken from the LW wiki):

 

When people talk about "AI", they're talking about an incredibly wide range of possibilities. Having a word like "AI" is like having a word for everything which isn't a duck.


Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Psychological Unity of Humankind, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.