Friendship is Optimal has launched and is being published in chunks on FIMFiction.
Friendship is Optimal is a story about an optimizer written to "satisfy human values through friendship and ponies." I would like to thank everyone on LessWrong who came out and helped edit it. Friendship is Optimal wouldn't be what is today without your help.
Thank you.
Teaser description:
Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. Hanna has built an A.I. Princess Celestia and given her one basic drive: to satisfy everybody's values through friendship and ponies. And Princess Celestia will follow those instructions to the letter...even if you don't want her to.
Here is the schedule for the next chapters:
Friday (Nov. 16th): Chapter 4 - 5
Monday (Nov. 19th): Chapter 6 - 7
Thursday (Nov. 22th): Chapter 8 - 9
Sunday (Nov. 25th): Chapter 10 - 11, Author's Afterword
It probably doesn't help that Celestia implies "it wouldn't be you" when explaining why Hanna uploaded. If the shut-down authority was tied to her biological body, then Celestia fails to say so, and talks instead about identity. If it was tied to her name, then conflating that with the uploading is misleading. If the point of uploading was to protect her against coercion, then that would be sufficient even without any change in authority, and "Hanna no longer exists" / "Now that she is not Hanna" are misleading. Either way, I endorse the pattern theory of identity, but I don't see any plausible way to interpret that exchange in support of it.
He's right, you know? That "Hanna is dead" line is kind of counter-productive. Did you really need to have Celestia act like such a dick to keep conflict and ambiguity?