If only I had programmed the robot to be more careful what I wished for.
This may be the best description of the Friendliness problem in popular media.
Watched it. Successful in getting the idea across, I think. Next stop: an intelligence explosion episode!
Looks like it's going to be in the season finale: http://en.wikipedia.org/wiki/Futurama_(season_6)#Episodes ("Overclockwise")
Somebody remind me why we can't have nanobots that build a different kind of nanobot that builds a different kind of nanobot that builds a different kind of nanobot that does all the work.
The depiction of the personalities of the individual 'mini/micro/nano benders' was interesting, They seemed to maintain the same independent personality, but cooperate because of shared goals. Also interesting as while an individual bender is not explicitly 'unfriendly' (just annoying) because they all pioritise their survival above other things, and desire to reduce their workload the overall effect is very harmful. So good example of unintended consequences in AI design there.
[Also favourite quote: "THE SEQUENCE IS NON-CONVERGING"]
I'm guessing the episode is better done than the Gargoyles episode (Season 2 Episode 33, "Walkabout").
Apropos the Wikipedia article, in what way is grey goo a "transhumanist theory"?
Grey goo scenarios are relatively straightforward extrapolations of mundane technological progress and complex system dynamics with analogues in real biological systems. Subscribing to transhumanism is not a prerequisite to thinking that grey goo is a plausible region of the technological development phase space.
Futurama, Season 6, Episode 15: Benderama
Naturally, this episode's goal is entertainment, not so much the accurate portrayal of the technology.
Quoting from wikipedia article:
The episode is not officially available for viewing online (but you can still find it, of course).