Friendship is Optimal has launched and is being published in chunks on FIMFiction.
Friendship is Optimal is a story about an optimizer written to "satisfy human values through friendship and ponies." I would like to thank everyone on LessWrong who came out and helped edit it. Friendship is Optimal wouldn't be what is today without your help.
Thank you.
Teaser description:
Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. Hanna has built an A.I. Princess Celestia and given her one basic drive: to satisfy everybody's values through friendship and ponies. And Princess Celestia will follow those instructions to the letter...even if you don't want her to.
Here is the schedule for the next chapters:
Friday (Nov. 16th): Chapter 4 - 5
Monday (Nov. 19th): Chapter 6 - 7
Thursday (Nov. 22th): Chapter 8 - 9
Sunday (Nov. 25th): Chapter 10 - 11, Author's Afterword
This was good. Probably the biggest problem was the dialogue. The human characters' lines often didn't sound like anything you'd imagine an actual human saying aloud in the situation. The parts with the marketing exec were probably the weakest in this regard. Like chapter 5, once someone learns that his AI has started doing large scale human uploading behind his back, he's probably going to do something involving a fire axe and the server room, not continue with philosophical dialogue. But the thing maintains a creepy ambiguity right to the end, and the pony thing plays right into this. Though could've maybe framed the dilemma a bit more in terms of "everyone's going to spend the rest of the eternity singleton-locked into a gamified virtual world with the level of cultural complexity appreciated by a clever 10-year-old and modified to love it with all their heart" instead of "but some people don't like ponies".
I would like it even if "but some people don't like ponies" got some air time. Or "but some people don't like to live in the simulation". Not necessarily haters or religious fanatics, just... I think the majority of people like their lifestyles and would like to preserve them, thankyouverymuch. Once it becomes clear that large scale human uploading is going on, they should have tried "something involving a fire axe", like you said.
I was surprised that it never happened, and Iceman failed to answer my questions so far as to why. Maybe he just ran out of steam and needed to get the main idea across?
What's "singleton-locked"?