Good post!
In this metaphor, it seems to me that Qui-Gon also has a strong positive prior on [a child with a lot of midi-chlorian] being good. In other word, because of the prophecy, he believes that developing a lot of capabilities is likely to turn out good. The council doesn't seem to share this prior. As you say, they are worried about the misalignement and the bad consequences it may lead to.
Of course, this is only a fiction, the fact that this kind of optimism led to some terrible outcome says little about how our future will be. A bad outcome made for a better story.
Thanks!
Yup I think you're right about Qui-Gon and the council having different priors here and that being the reason for their different reactions. And yeah, definitely gotta be careful about The Logical Fallacy of Generalizing from Fictional Evidence here.
In Star Wars: Episode I - The Phantom Menace, Master Jedi Qui-Gon Jinn discovers Anakin. Anakin is just a young child slave on the forgettable planet of Tatooine. However, Qui-Gon feels something noteworthy about the boy.
He feels... the Force. To confirm this feeling, he takes a sample of blood from Anakin.
Anakin: What are you doing?
Qui-Gon: Checking your blood for infections. Run along. You have a big day tomorrow. Sleep well Anny.
Qui-Gon: Obi-Wan.
Obi-Wan: Yes master.
Qui-Gon: I need an analysis of this blood sample I'm sending you.
Obi -Wan: Wait a minute.
Qui-Gon: I need a midi-chlorian count.
Obi-Wan: The reading is off the chart. Over 20,000. Even Master Yoda doesn't have a midi-chlorian count that high.
Qui-Gon: No Jedi has.
Obi-Wan: What does that mean?
Qui-Gon: [dramatic pause]
Qui-Gon: I'm not sure.
So then, Qui-Gon wants to take advantage of this. He basically has a budding superweapon in front of him and wants to utilize it in the name of Good.
More specifically, he believes that Anakin is the Chosen One.
An ancient Jedi prophecy, made by an ancient Jedi mystic and recorded on a holocron, foretold that the Chosen One would restore balance to the Force by destroying the Sith. The prophecy also predicted that the Chosen One would also become one of the strongest Force-sensitives in the galaxy, rivaling the power of the most legendary champions of the Jedi Order.
Although the Jedi had long spoken of balance in the Force, there was no consensus on what that meant for the galaxy. Some people thought that the light side of the Force and the dark side of the Force had to be equal in order to create balance, while others believed that the light could be the Force in its natural state, making the dark side the cause of imbalance. In contrast to the Jedi's view of balance, the Sith Lord Darth Sidious believed that the dark side needed to be free, and the light side suppressed, in order to achieve balance in the Force.
Given this, Qui-Gon wants to train Anakin as a Jedi so that the prophecy can be fulfilled. But when Qui-Gon presents this proposal to the Jedi Counsel, they are not eager to jump on it.
Yoda: Master Qui-Gon. More to say, have you?
Qui-Gon: With your permission my master, I have encountered a vergence in the Force.
Yoda: A vergence, you say?
Mace Windu: Located around a person?
Qui-Gon: A boy. His cells have the highest concentration of midi-chlorians I have seen in a life form. It is possible he was conceived by the midi-chlorians.
Mace Windu: You refer to the prophecy of the one who will bring balance to the Force. You believe it's this boy?
Qui-Gon: I don't presume to...
Yoda: But you do. Revealed your opinion is.
Qui-Gon: I request the boy be tested, Master.
Yoda: Oh, trained as a Jedi you request for him, hmm?
Qui-Gon: Finding him was the will of the Force. I have no doubt of that.
Yoda and Mace Windu look at each other with hesitation and exasperation.
Mace Windu: Bring him before us, then.
Yoda and Mace Windu stare into space, looking serious and concerned.
Damn Yoda. What a good Bayesian.
New scene.
Obi-Wan: The boy will not pass the council's test, Master. He's too old.
Qui-Gon: Anakin will become a Jedi, I promise you.
Obi-Wan: Do not defy the council Master, not again.
Qui-Gon: I shall do what I must, Obi-Wan.
Obi-Wan: If you would just follow the code, you would be on the council. They will not go along with you this time.
Qui-Gon: You still have much to learn my young apprentice.
New scene.
Anakin: A ship. A cup. A ship. A speeder.
Yoda: Hmm. How feel you?
Anakin: Cold, sir.
Yoda: Afraid are you?
Anakin: No, sir.
Yoda: See through you we can.
Mace Windu: Be mindful of your feelings.
Ki-Adi-Mundi: Your thoughts dwell on your mother.
Anakin: I miss her.
Yoda: Mmm. Afraid to lose her I think, mmm?
Anakin: What does that got to do with anything?
Yoda: Everything. Fear is the path to the dark side. Fear leads to anger. Anger leads to hate. Hate leads to suffering. I sense much fear in you.
And so, the Jedi Counsel declines to allow Anakin to be trained as a Jedi.
When I think about these scenes, I think about alignment vs capabilities.
In the context of AI, we talk about alignment and capabilities. Capabilities research is what makes AIs more powerful. It's what takes us from GPT-2 to GPT-3 to GPT-4 to GPT-N. Alignment is what increases the probability of the AI doing good instead of bad.
In the context of Star Wars, Qui-Gon seems to be ignoring the question of alignment. He has faith that, if trained, Anakin's power would be used for good. That it would be a net positive. The Jedi Counsel on the other hand reacts with fear, worry, skepticism, and a desire to explore the question of alignment before pursuing developments in capabilities.
This question of alignment vs capabilities is not specific to AI and Star Wars, of course. It is more general than that. It can be applied to anything, really.
Knowledge of the physics of subatomic particles lead to both nuclear energy and nuclear weapons. Knowledge of the chemistry of molecules lead to both medicine and cigarettes. Knowledge of human psychology lead to both cognitive behavioral therapy and Facebook.
It's not just knowledge though. An apple can be consumed as a delicious snack (good) or thrown at the head of your younger brother (bad).
If things can be used for both good or for bad, surely we have to think about whether or not the good will outweigh the bad. I don't think this happens nearly enough though. We currently have too much Qui-Gon and not enough Yoda.