637

LESSWRONG
LW

636
AI RiskOuter AlignmentAIWorld Modeling
Frontpage

9

"Sorcerer's Apprentice" from Fantasia as an analogy for alignment

by awg
29th Mar 2023
1 min read
4

9

This is a linkpost for https://video.disney.com/watch/sorcerer-s-apprentice-fantasia-4ea9ebc01a74ea59a5867853

9

"Sorcerer's Apprentice" from Fantasia as an analogy for alignment
6Shmi
3sab
1awg
1benjamin.j.campbell
New Comment
4 comments, sorted by
top scoring
Click to highlight new comments since: Today at 7:13 AM
[-]Shmi2y60

Yeah, this example is excellent and is being brought up repeatedly on this site and elsewhere. Here is Gary Marcus in 2012:

https://www.newyorker.com/news/news-desk/moral-machines (https://archive.is/1Q7zp)

Reply
[-]sab2y30

FWIW this story is much older than Mickey: https://en.wikipedia.org/wiki/The_Sorcerer's_Apprentice

Reply
[-]awg2y10

Totally! Just figured for many like myself this might have been the first/only exposure to the tale.

Reply
[-]benjamin.j.campbell2y10

And it goes deeper. Because what if Mickey never actually woke up, and the brooms had been keeping him asleep the whole time? The Sleeping Beauty problem is actually present in quite a lot of Disney media where the MC goes to sleep. It's also a theme in Mulan. Maybe she never went to war and made her parents proud. It may well have been a dream she just didn't wake up from.

Reply
Moderation Log
More from awg
View more
Curated and popular this week
4Comments
AI RiskOuter AlignmentAIWorld Modeling
Frontpage

The story is simple: Mickey is an apprentice to a powerful sorcerer whose magic comes from his hat. Mickey is tasked with carrying buckets of water up a long flight of stairs and dumping them into a basin–hard work for a mouse! When the sorcerer steps out, however, Mickey takes his hat and uses its magic to enchant a broomstick to do the water carrying for him. Aha! Mickey is able to rest now, and he falls asleep to dream about all the wonderful magic he can do. But he awakes to a terrible discovery: the enchanted broomstick won't stop carrying and dumping water and the basin is now overflowing! Mickey attempts to alter the broomstick's course of action, but he can't; it's too late! So he tries to hack it to pieces, thinking that will solve the problem. But no, the enchantment enables the broomstick to make copies of itself from the hacked pieces! Now there are even MORE of them doing the unwanted task, and Mickey, try as he might, can't possibly hope to overcome them. It's not until the sorcerer returns (the one who truly knows how to wield the magic hat) that all is put to rights.

I thought it fascinating how this popular story from when we were all little actually encapsulates the alignment problem pretty neatly. Just wanted to share!