I wasn't involved in the editing process, so I don't know if you address this in future chapters, but it seems as though most people who are commenting on Equestria Daily are rejecting the concept of uploading because it would be "like dying" (similar to the Star Trek transporter thought experiment). I hope that this fear is addressed comprehensively because it seems like a major deterrence for people who would otherwise be supportive of the idea.
Also, I think that the people who "know how the story ends" because they've read about robot apocalypse analogues before are entirely missing the point, and I hope that gets expressed in the story somewhere as well.
And (as a history and archiving dork), I hope that people don't forget what came before the utopia. To lose that much human history would be devastating for me. Although I'm not sure if it goes against Celestia's programming -- people could know about the time before ponies and still have their values maximized through friendship and ponies.
(I haven't submitted it to EQD, though I plan to once I've posted all the chapters. I'm going to assume you mean FIMFiction.)
It's a bit frustrating because, to me, this came out of nowhere. The idea that "it wouldn't be you" isn't something I thought would be a problem, probably because I believe the exact opposite and assumed everyone else would. After reading so much LW, materialism and a computational theory of mind are just background assumptions.
So: ooops.
I started out thinking about making an edit to Lars and Celestia's conversation where he's asking if it's legal to destroy a human body, but after discussing it with a few people, I decided against it. If I simply have Celestia say that it isn't murder, that it's obviously not a clone and is the same individual, I don't think that's going to convince anyone. Especially since it'd be a bad idea to put the audience's complaints on Lars, who isn't supposed to be sympathized with. I don't really think there's much I can do here.
The idea that "it wouldn't be you" isn't something I thought would be a problem
It probably doesn't help that Celestia implies "it wouldn't be you" when explaining why Hanna uploaded. If the shut-down authority was tied to her biological body, then Celestia fails to say so, and talks instead about identity. If it was tied to her name, then conflating that with the uploading is misleading. If the point of uploading was to protect her against coercion, then that would be sufficient even without any change in authority, and "Hanna no longer exists" / "Now that she is not Hanna" are misleading. Either way, I endorse the pattern theory of identity, but I don't see any plausible way to interpret that exchange in support of it.
He's right, you know? That "Hanna is dead" line is kind of counter-productive. Did you really need to have Celestia act like such a dick to keep conflict and ambiguity?
Fanfanfiction idea: during the months of CelestAI's world takeover, a foresightful Hasbro VP races to get My Little Pony Equestria Girls made and released so that humans will be canon and people don't have to be ponies for eternity. Title: Friendship Is Optional.
The moral tension of what Celestia was doing was well done. I found myself getting excited at how awesome everything was, and then muttering aloud about the "creeping horror" that was coming. I also loved the parallels to the AI-boxing experiments, except the Princess can argue people into boxing themselves.
Thanks very much for writing this! In lieu of anything specifically useful, I leave you with a small proportion of the warm fuzzies you provided to me :)
Any plans to post this elsewhere? I feel like some readers will ignore anything on a site with as many pictures of ponies as fimfiction.
There are upsides to having a story in one central location, however reaching fewer readers than would be possible otherwsie is a high price to pay for that.
I suggest that alternative place should provide readership statistics (as FiMFiction does) and include a note somewhere that canonical place is at FiMFiction (so that discussion will take place there)
OTOH, I would like to know how or where higher levels of discourse can be inspired. Unfortunately, on FiMFiction pictures remain legal tender for comments and there are many shallow comments ("uploading is murder" and the like).
Has it changed significantly since you first released it as a beta?
I read it and commented at that time, so I'm curious if it was mostly minor changes (eg spelling/grammar/wording) or changes to the plot.
Mostly minor changes, but the prologue was completely rewritten. There were some changes to details about how she grew at the Celestial level towards the end.
The plot and general direction remain the same.
This was good. Probably the biggest problem was the dialogue. The human characters' lines often didn't sound like anything you'd imagine an actual human saying aloud in the situation. The parts with the marketing exec were probably the weakest in this regard. Like chapter 5, once someone learns that his AI has started doing large scale human uploading behind his back, he's probably going to do something involving a fire axe and the server room, not continue with philosophical dialogue. But the thing maintains a creepy ambiguity right to the end, and the pony thing plays right into this. Though could've maybe framed the dilemma a bit more in terms of "everyone's going to spend the rest of the eternity singleton-locked into a gamified virtual world with the level of cultural complexity appreciated by a clever 10-year-old and modified to love it with all their heart" instead of "but some people don't like ponies".
I would like it even if "but some people don't like ponies" got some air time. Or "but some people don't like to live in the simulation". Not necessarily haters or religious fanatics, just... I think the majority of people like their lifestyles and would like to preserve them, thankyouverymuch. Once it becomes clear that large scale human uploading is going on, they should have tried "something involving a fire axe", like you said.
I was surprised that it never happened, and Iceman failed to answer my questions so far as to why. Maybe he just ran out of steam and needed to get the main idea across?
What's "singleton-locked"?
I respectfully request an explanation of the merits of publishing like this when you have the entire fic written already.
Three reasons:
Most people who read the beta version know why I specifically want chapter 4 to be released on a Friday. It was serendipitous and unplanned when I released the beta on a Friday evening, and I'm trying to recapture that in the real release. Making sure this happened was a design goal of my schedule.
My second roomate tried to read the entire thing in one sitting, and complained that once things got heavy and philosophical, he felt a bit overwhelmed. I want people to have some time to mull things over and think "what would I do in that situation?" before they see what happens next.
One of the Equestria Daily prereaders recommended that I do so. If I had written this chapter by chapter and posted as I went, I would get n chances on the new list on FIMFiction, but if I post the entire thing at once, I only get 1. Each time Friendship is Optimal is on the new list is a chance to introduce it to new readers. Posting in chunks corrects for a bias against writing and editing everything up front, since, IIUC, you get less attention than if you post as you go. Putting three days between releases is a trade off between not making people too antsy for the next chapter and trying to give other stories a chance on the new and featured lists.
For the benefit of those who are interested in the story, but aren't tracking it in every possible way:
There's a work-in-progress document called Rules of the Optimalverse which should be interesting reading if you think about writing a story in this universe, would like to pick Iceman's mind on the subjects of the story and/or don't get enough fiction with embedded comments lately :)
And a FiMFiction group called The Optimalverse which is formally for those aspiring to write side stories to Friendship is Optimal, but I guess those interested in the story itself should go there too.
At this point, I had already taken over his computer and analyzed the virus. It was highly contagious, and would lock muscles in the jaw into a permanent smile, but otherwise wouldn’t harm the host. It would have quickly spread to every country in the world, except for Madagascar.”
Wait... is this a reference to the Pandemic game? If so, kudos.
EDIT: Finish chapter, look at comments, learn that I am one of dozens, nay, several to say that. Oh well.
Everyone in the comments loves that reference. Everyone. (be it Google Docs version that was only linked from Less Wrong, or FiMFiction version)
I just finished the story. I think the main idea is obviously good, but I find the execution too simplistic. The princess’s intelligence comes off as human in her social interactions (not superhuman), and humans seem dumbed down. I don’t think the AI could have forced everyone to “upload” with the level of coercion shown in the novel.
I myself would not have uploaded unless I was already dying. I see no benign reason that the process of uploading needs to be destructive.
I also doubt people with real power would be easily swayed to upload, e.g., the original programmer. In the end, people’s values are in conflict, and there could be alien powers, etc. There is every reason to believe that whoever still controls some resources in the “real” world will still be able to “satisfy” their values more than the powerless ponies.
I confess waiting for the chapters to be released is more annoying then anything, especially as the slowness is deliberate versus just waiting for the chapter to be written then released.
While I understand the rational in reaching a larger reader pool, I confess that I myself have unfavorited and unfollowed the fic (not permanently) till the 25th and its best if I pretend it does not exist until then when the entire thing is out there.
Books are meant to be read in a single sitting, not consumed as piecemeal (unless no other option exists). It's literature, not a comic or an episodic tv series after all.
I may be hyped on Ponies because I've just finished reading Fallout Equestria two weeks back, but I really liked the first three chapters of this, can't wait to read the next ones.
This was a fun read! Two quick comments about Chapter 11. First, there is a "Euclidian" which should be a "Euclidean."
Second, I have a mild technical objection to your description of Equestria-space as not being commutative. "Noncommutative geometry" has a mathematical meaning (it is not completely precise yet because the field is relatively young), and it refers to something different, namely coordinates not being commutative (e.g. position and momentum in quantum mechanics). What you're describing is more like a Cayley graph of a noncommutative group. The bare graph itself has no notion of commutativity or noncommutativity: it's the extra fact that there are six specific ways to go from a block to one of its neighbors that look like elements of Z^3 for familiarity but that are actually elements of the free group on 6 generators or some quotient thereof.
I've fixed the spelling error.
If you could suggest a phrasing that's both accurate, but also short and won't require the reader to know much graph theory. I think I'm pushing it as it is right now. I said "noncommutative" since I assumed that basically my entire audience would have been exposed to commutative binary function from their high school proofs classes.
"Noncommutative" should be applied to the operations (up, left, etc.) rather than to the space. Where are you that it's typical for there to be classes on proofs in high school?
I went to high school in the United States and had a semester in proofs during high school. But now that you mention it, this may not be common and I might be generalizing from non-representative schools...
My understanding of what's typical in American high schools is that most students only get as far as trigonometry or precalculus. Stronger students will take some form of calculus. But even at that point the closest thing a typical student will come to taking a course on proofs is seeing some "two-column proofs" of statements in Euclidean geometry.
Friendship is Optimal has launched and is being published in chunks on FIMFiction.
Friendship is Optimal is a story about an optimizer written to "satisfy human values through friendship and ponies." I would like to thank everyone on LessWrong who came out and helped edit it. Friendship is Optimal wouldn't be what is today without your help.
Thank you.
Teaser description:
Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. Hanna has built an A.I. Princess Celestia and given her one basic drive: to satisfy everybody's values through friendship and ponies. And Princess Celestia will follow those instructions to the letter...even if you don't want her to.
Here is the schedule for the next chapters:
Friday (Nov. 16th): Chapter 4 - 5
Monday (Nov. 19th): Chapter 6 - 7
Thursday (Nov. 22th): Chapter 8 - 9
Sunday (Nov. 25th): Chapter 10 - 11, Author's Afterword