Hmmm... and I thought you were going to suggest that, if you succeded in making an AI that you must be god. I would've loved to be there to offer him that option instead. LOL!
Of course, even if what you said is what we really mean, I'm not sure which one is more effective at getting people to think, but your story shows that it's usually good (and at least entertaining) to try being more direct, every once in a while. I just find it easier to break through the social convention of politeness with humor.
I am quite impressed at your capability of signaling your prodigious intelligence. Less pompously, moments like that make for fond memories.
Nice job, but the mention of Aumann's theorem looks a bit like a sleight of hand: did the poor fellow ever learn that the theorem requires the assumption of common priors?
Robin sort-of generalized it so that it doesn't. http://www.overcomingbias.com/2006/12/why_common_prio.html
My big question though is whether this exchange led to a lasting change in the fellow's opinion as to the possibility of AI. In practice it seems to me that most of the time when people decisively loose an argument they still return to their original position within a few days just by ignoring that it ever happened.
This story is related to the phenomena whereby the most intelligent and educated religious folks are very careful to define their beliefs so that there can be no conflict with observations, while ordinary people are more prone to allow their religion to have implications, which are then subject to challenges like Eliezer's. It is fun to pick holes in the less educated views, but to challenge religion overall it seems more honest to challenge the most educated views. But I usually have trouble figuring out just what it is that the most educated religious folks think exactly.
I've mentioned before that my attempt to salvage a belief in God ultimately resulted in something like H. P. Lovecraft's Azathoth, which might not be too surprising as it was that ardent materialist's parody of the God of the Old Testament.
A few questions and comments:
1) What kind of dinner party was this? It's great to expose non-rigorous beliefs, but was that the right place to show off your superiority? It seems you came off as having some inferiority complex, though obviously I wasn't there. I know that if I'm at a party (of most types), for example, my first goal ain't exactly to win philosophical arguments ...
2) Why did you have to involve Aumann's theorem? You caught him in a contradiction. The question of whether people can agree to disagree, at least it seems to me, is an unnecessary distraction. And for all he knows, you could just be making that up to intimidate him. And Aumann's Theorem certainly doesn't imply that, at any given moment, rectifying that particularly inconsistency is an optimal use of someone's time.
3) It seems what he was really trying to say was someting along the lines of "while you could make an intelligence, its emotions would not be real the way humans' are". ("Submarines aren't really swimming.") I probably would have at least attempted to verify if that's what he meant rather latching onto the most ridiculous meaning I could find.
4) I've had the same experience with people who fervently hold beliefs but don't consider tests that could falsify them. In my case, it's usually with people who insist that the true rate of inflation in the US is ~12%, all the time. I always ask, "so what basket of commodity futures can I buy that consistently makes 12% nominal?"
That was cruel. Fun, but cruel.
A woman who had stood nearby, listening to the conversation, said to me gravely, "That was beautiful."
And people wonder why men argue so often.
Meanwhile, over at the next table, there was the following conversation:
"I believe science teaches us that human-caused global warming is an urgent crisis."
"You mean if it's either not a problem or can be fixed easily, it proves science is false?"
Even if religion is divinely inspired, a person's understanding of one aspect of religion can be wrong without invalidating all of that person's other religious beliefs.
I know that if I'm at a party (of most types), for example, my first goal ain't exactly to win philosophical arguments ...
Funny, I've always thought that debates are one of the most entertaining forms of social interaction available. Parties with a lot of strangers around are one of the best environments for them - not only don't you know in advance the opinions of the others, making the discussions more interesting, but you'll get to know them on a deeper level, and faster, than you could with idle small talk. You'll get to know how they think.
Good catch, Pseudonymous. Robin, my guess is that they're crypto-skeptics, performing for their self-perceived comparative economic/social advantage. Eliezer, please don't make something that will kill us all.
"Funny, I've always thought that debates are one of the most entertaining forms of social interaction available."
We may not have rationality dojos, but in-person debating is as good an irrationality dojo as you're going to get. In debating, you're rewarded for 'winning', regardless of whether what you said was true; this encourages people to develop rhetorical techniques and arguments which are fully general across all possible situations, as this makes them easier to use. And while it may be hard to give public demonstrations of rationality, demonstrations of irrationality are easy: simply talk about impressive-sounding nonsense in a confident, commanding voice, and people will be impressed (look at how well Hitler did).
I think the idea of argument is to explore an issue, not "win" or "lose". If you enter an argument with the mentality that you must be right, you've rather missed the point. There wasn't an argument here, just a one-sided discussion. It was a bludgeoning by someone with training and practice in logical reasoning on someone without. It was both disgusting and pathetic, no different than a high-school yard bully pushing some kid's face in the dirt because he's got bigger biceps. Did the outcome of this "argument" stroke your ego?
All-in-all, I'm not sure this is a story you should want to share. To put this in uncomplicated terms, it makes you sound like a real a$$hole.
(FYI, the MRA who posted is not Ames.)
MRA, the difference between winning an argument with someone, versus pushing them into the dirt - well, there's a number of differences, really. The three most important are: First, I didn't force him to talk to me. Second, losing an argument makes you stronger. (Or rather, it gives you a precious chance to become stronger; whether he took advantage of it was up to him. Winning is a null-op, of course.)
Third and above all, in factual arguments there is such a thing as truth and falsity.
"I believe science teaches us that human-caused global warming is an urgent crisis." "You mean if it's either not a problem or can be fixed easily, it proves science is false?" Science has been proved false many times. Those things proven to be false are no longer science. OTOH most religious beliefs are dogmatic. They can't be discarded from that religion without divine intervention/prophecy.
Where do people get the impression that we all have the right not to be challenged in our beliefs? Tolerance is not about letting every person's ideas go unchallenged; it's about refraining from other measures (enforced conformity, violence) when faced with intractable personal differences.
As for politeness, it is an overrated virtue. We cannot have free and open discussions, if we are chained to the notion that we should not challenge those that cannot countenance dissent, or that we should be free from the dissent of others. Some people should be challenged often and publicly. Of course, the civility of these exchanges matters, but, as presented by Eliezer, no serious conversational fouls or fallacies were committed in this case (contemptuous tone, ad hominems, tu quoque or other Latinate no-nos, etc.).
Mark D,
How do you know what the putative AI "believes" about what is advantageous or logical? How do you know that other humans are feeling compassion? In other words, how you feel about the Turing test, and how, other than their behavior, would you be able to know about what people or AIs believe and feel?
We may not have rationality dojos, but in-person debating is as good an irrationality dojo as you're going to get. In debating, you're rewarded for 'winning', regardless of whether what you said was true
Only if you choose to approach it that way.
Johnny Logic: some good questions.
“Tolerance is not about letting every person's ideas go unchallenged; it's about refraining from other measures (enforced conformity, violence) when faced with intractable personal differences.”
That’s certainly the bare minimum. His beliefs have great personal value to him, and it costs us nothing to let him keep them (as long as he doesn’t initiate theological debates). Why not respect that?
“How do you know what the putative AI "believes" about what is advantageous or logical?”
By definition, wouldn’t our AI frie...
Mark M.,
"His beliefs have great personal value to him, and it costs us nothing to let him keep them (as long as he doesn’t initiate theological debates). Why not respect that?"
Values may be misplaced, and they have consequences. This particular issue doesn't have much riding on it (on the face of it, anyway), but many do. Moreover, how we think is in many ways as important as what we think. The fellows ad hoc moves are problematic. Ad hoc adjustments to our theories/beliefs to avoid disconfirmation are like confirmation bias and other fallacie...
JL, I’ve programmed in several languages, but you have me correctly pegged as someone who is more familiar with databases. And since I’ve never designed anything on the scale we’re discussing I’m happy to defer to your experience. It sounds like an enormously fun exercise though.
My original point remains unanswered however. We’re demanding a level of intellectual rigour from our monotheistic party goer. Fair enough. But nothing I’ve seen here leads me to believe that we’re as open minded as we’re asking him to be. Would you put aside your convictions and a...
Mark D,
"JL, I’ve programmed in several languages, but you have me correctly pegged as someone who is more familiar with databases. And since I’ve never designed anything on the scale we’re discussing I’m happy to defer to your experience. It sounds like an enormously fun exercise though."
There are programs (good ol' chatter bots) that use methods like you supposed, but they are far from promising. No need to defer to me-- I am familiar with machine learning methods, some notable programs and the philosophical debate, but I am far from an expert ...
A less personal response to the second bit I quoted from Mark D: Yes, changing our beliefs in the face of good evidence and argument is desirable, and to the extent that we are able to do this we can be called critical thinkers.
Would you put aside your convictions and adopt religion if a skilful debater put forward an argument more compelling than yours?
To the extent the answer is "No" my atheism would be meaningless. I hope the answer is "Yes", but I have not been so tested (and do not expect to be; strong arguments for false theses should not exist).
First: The argument wasn't the author being an a$$hole. He was stating the nature of his business, which is a very normal thing to do at a social gathering. (We are, to a disturbing extend, defined by our income.) Godboy dismissed his profession as quixotic, leading the author to the notion that if he created a working AI, that it would disprove God, in the mind of his coparticipant in discussion. This was a logical inferrence, based on the statement that inspired it.
Second: The only winner in a conversation is the person who learns something. I believe, t...
Wouldn't it be easier to say, an AI is not a soul? In what sense do these two words have the same meaning? An AI is a non-existant entity which, due to the unflagging faith of some, is being explored. A soul is an eternal being granted life (human only?) by god (should that be capitalized?) Comparing them is what leads to the problem.
Douglas, (1) what makes you think that anyone was suggesting that "AI" and "soul" have the same meaning?, (2) in what way would "an AI is not a soul" be a useful substitute for anything else said in this discussion?, and (3) why should comparing the two notions lead to any problems, and in particular to whatever you're calling "the problem" here?
I don't think it's any more obvious that there are no AIs than that there are no souls. That is: perhaps, despite appearances, my computer is really intelligent, and thinks i...
g- the man said, "I don't believe AI is possible because only God can make a soul." "...If I can make an AI it proves your religion false?" Somebody in this exchange has equated the making of an AI with the making of a soul. That's why I would suggest that the words have been confused. An AI is not a soul would be useful in this discussion because it would clarify that the making of one would not invalidate the existence of the other or the statement that "only God can make a soul". Comparing the two notions would not be a pr...
Douglas: OK, I hadn't realised you were talking about him; my bad. And, sure, another approach Eliezer could have taken is to say "an AI and a soul aren't the same thing". But I don't see why that would be any improvement on what he actually did do.
Also: "soul" is used vaguely enough that I don't think Eliezer could justifiably claim that an AI wouldn't have to be a soul. If his interlocutor believed, e.g., that a soul is what it takes in order to have real beliefs, feelings, will, etc., then saying "oh no, I'm not talking about so...
g- you ask good questions. My point about AI and religion is that rather than pretending that one is related to the other, AI would benefit from clearing up this confusion. (So would the religious) Perhaps the way Elizer went about it was OK I would define "soul" as a non-corporeal being that exists separable from the body and that survives body death. (I want to say something about the soul being the true source of consciousness and ability-- OK, I said it)
Seems more like Aikido. I sense a broken spirit more than a redirection of thought or processes of it and his belief system. Simply put, honey has always gotten me more flies than vinegar.
Rereading the post, I don't understand why the fellow didn't just say "I defy your ability to build an AI" in response to your first question. Maybe he was intimidated at the moment.
I have to heartily disagree with those that seem to think it impolite to disagree with the religious. Remember this same person is going to go out and make life and death decisions for himself and others. Notice also that it was the theist who started the debate.
All you did was show that your argumentative skills were better. His intial belief mentioned souls, and i dont think you ever did. I'd like to see some sort of testability for souls :)
As to your reply of possibly proving his religion false, if he was better at arguing, he may have replied that at the least it might prove his understanding of religion false.
And of course its not as if you have created an AI.
I attended a lecture by noted theologian Alvin Plantinga, about whether miracles are incompatible with science. Most of it was "science doesn't say it's impossible, so there's still a chance, right?"-type arguments. However, later on, his main explanation for why it wasn't impossible that God could intervene from outside a closed system and still not violate our laws of physics was that maybe God works through wavefunction collapse. Maybe God creates miracles by causing the right wavefunction collapses, resulting in, say, Jesus walking on water, rising from the dead, unscrambling eggs, etc.
Recalling this article, I wrote down and asked this question when the time came:
..."The Many-Worlds Interpretation is currently [I said "currently" because he was complaining earlier about other philosophers misrepresenting modern science] one of the leading interpretations of quantum mechanics. The universe splits off at quantum events, but is still deterministic, and only appears probabilistic from the perspective of any given branch. Every one of the other branches still exists, including ones where Jesus doesn't come back. If true, how does this affect your argument?&q
Now, what I don't get is why he let you force him to change his position. If he really believed that it was impossible for you to create AI, why wouldn't he have just said "yes," and then sit back, comfortable in his belief that you will never create an AI?
This post's presence so early in the core sequences is the reason I nearly left LW after my first day or two. It gave me the impression that a major purpose of rationalism was to make fun of other people's irrationality rather than trying to change or improve either party. In short, to act like a jerk.
I'm glad I stuck around long enough to realize this post wasn't representative. Eliezer, at one point you said you wanted to know if there were characteristically male mistakes happening that would deter potential LWers. I can't speak for all women, but this post exemplifies a kind of male hubris that I find really off-putting. Obviously the woman in the penultimate paragraph appreciated it in someone else, but I don't know if it made her think, "This is a community I want to hang out with so I, too, can make fools of other people at parties."
If I were the host I would not like it if one of my guests tried to end a conversation with "We'll have to agree to disagree" and the other guest continued with "No, we can't, actually. There's a theorem of rationality called Aumann's Agreement Theorem which shows that no two rationalists can agree to disagree." In my book this is obnoxious behavior.
Having fun at someone else's expense is one thing, but holding it up in an early core sequences post as a good thing to do is another. Given that we direct new Less Wrong readers to the core sequence posts, I think they indicate what the spirit of the community is about. And I don't like seeing the community branded as being about how to show off or how to embarrass people who aren't as rational as you.
What gave me an icky feeling about this conversation is that Eliezer didn't seem to really be aiming to bring the man round to what he saw as a more accurate viewpoint. If you've read Eliezer being persuasive, you'll know that this was not it. He seemed more interested in proving that the man's statement was wrong. It's a good thing to learn to lose graciously when they're wrong, and learn from the experience. ...
Shortly after my stroke, my mom (who was in many ways more traumatized by it than I was) mentioned that she was trying to figure out what it was that she'd done wrong such that God had punished her by my having a stroke. As you might imagine, I contemplated a number of different competing responses to this, but what I finally said was (something along the lines of) "Look, I understand why you want to build a narrative out of this that involves some responsible agent making decisions that are influenced by your choices, and I recognize that we're all in a difficult emotional place right now and you do what you have to do, but let me offer you an alternative narrative: maybe I had a survivable stroke at 40 so I'd start controlling my blood pressure so I didn't have a fatal one at 45. Isn't that a better story to tell yourself?"
I was pretty proud of that interaction.
If I were the host I would not like it if one of my guests tried to end a conversation with "We'll have to agree to disagree" and the other guest continued with "No, we can't, actually. There's a theorem of rationality called Aumann's Agreement Theorem which shows that no two rationalists can agree to disagree." In my book this is obnoxious behavior.
I'd find it especially obnoxious because Aumann's agreement theorem looks to me like one of those theorems that just doesn't do what people want it to do, and so ends up as a rhetorical cudgel rather than a relevant argument with practical import.
Agreed. If this was Judo, it wasn't a clean point. EY's opponent simply didn't know that the move used on him was against the sport's rules, and failed to cry foul.
Storytelling-wise, EY getting away with that felt like a surprising ending, like a minor villain not getting his comeuppance.
Interesting. When I am arguing with somebody, I usually get them to explicitly define every one of their terms and then use the definitions to logic them into realising that their argument was faulty. A more rational person could have escaped your comment simply by defining AI as human-like intelligence- ie, ability to create, dream, emote and believe without prior programming for those things. And yes, I am religious and my belief can be overturned by proof. If aliens are found with human-like intelligence, I will give up my faith entirely- but until then, just about anything else can be explained from within my ideology.
[ Disclaimer: This is my first post so please don't go easy on me. ]
After reading a handful of comments I am surprised to see so many people think of what Eliezer did here as some sort of "bad" thing. Maybe I'm missing something but after reading all I saw was him convincing the man to continue the discourse even though he initially began to shy away from it.
Perhaps citing a theorem may have intimidated him a little, but in all fairness Eliezer did let him know at the outset that he worked in the field of Artificial Intelligence.
I've already seen plenty of comment here on just how awkward this post is to be so early in the Sequences, and how it would turn people off, so I won't comment on that.
However: Seeing this post, early in the sequences, led me to revise my general opinion of Eliezer down just enough that I managed to catch myself before I turned specific admiration into hero-worship (my early, personal term for the halo effect).
I seriously, seriously doubt that's the purpose of this article, mainly because if Eliezer wanted to deliberately prevent himself from being affecti...
There's a theorem of rationality called Aumann's Agreement Theorem which shows that no two rationalists can agree to disagree. If two people disagree with each other, at least one of them must be doing something wrong.
This seems like one of those things that can be detrimental if taught in isolation.
It may be a good idea to emphasize that only one person in a disagreement doing something wrong is far less likely than both sides in a disagreement doing something wrong.
I can easily imagine someone casually encountering that statement, and taking it to ins...
Eliezer, that's false reasoning. I'm not religious, so don't take this as the opening to a religious tirade, but it's a pet peeve of mine that intelligent people will assert that every belief within a religion is wrong if only one piece of it is wrong.
There are a billion and one reasons why a body of knowledge that is is mostly correct (not saying I think religions are) could have one flaw. This particular flaw doesn't prove God doesn't exist, it would only prove God souls aren't necessary for an intelligent life form to survive, or (perhaps, to a religi...
Me: Writes on hand "Aumann's Agreement Theorem". Thank you Eliezer, you have no idea how much easier you just made my Theory of Knowledge class. Half of our discussions in class seem to devolve into statements about how belief is a way of knowing and how everyone has a right to their own belief. This (after I actually look up and confirm for myself that Aumann's Agreement Theorem works) should make my class a good deal less aggravating.
I wrote a long post saying what several people had already said years ago, then shortened it. Still, because this post has made me mad for years:
1) Of COURSE people can agree to disagree! If not, EY is telling this guy that no two rationalists currently disagree about anything. If THAT were true, it's so fascinating that it should have derailed the whole conversation!
(Leaving aside, for the moment, the question of whether Aumann's theory "requires" a rationalist to agree with a random party goer. If it really did, then the party goer could convince EY by ...
While I understand the absolute primal urge to stomp on religious texts used to propagate compulsory heterosexuality, I do think this exchange ended up a bit of a poor game, when it seems like he'd be mostly interested in discussing how the emotions of programmed thought might differ from ours (and that's a fun well to splash around in, for a while)(though deposing of cult-friendly rhetoric is valuable too, even if you have to get nasty).
I'm mildly concerned about the Reign Of Terror Precept, but I also understand it. It's just disappointing to know that t...
I wonder if this post would have gotten a better reception if the stooge had been a Scientologist or a conspiracy theorist or something, instead of just a hapless normie.
Eliezer, that's false reasoning. I'm not religious, so don't take this as the opening to a religious tirade, but it's a pet peeve of mine that intelligent people will assert that every belief within a religion is wrong if only one piece of it is wrong.
There are a billion and one reasons why a body of knowledge that is is mostly correct (not saying I think religions are) could have one flaw. This particular flaw doesn't prove God doesn't exist, it would only prove God souls aren't necessary for an intelligent life form to survive, or (perhaps, to a religious person) that God isn't the only entity that can make them.
It's easy to get lazy when one's opponent isn't challenging enough (I've done it occasionally) and I've said stuff like that. I think it's best not to. They're not convincing to the opposition and we're not challenging ourselves to improve.
The framing of the first sentence gives me a desperately unfair expectation for the discussion inside HPMOR- I'm excited.
You can have some fun with people whose anticipations get out of sync with what they believe they believe.
I was once at a dinner party, trying to explain to a man what I did for a living, when he said: "I don't believe Artificial Intelligence is possible because only God can make a soul."
At this point I must have been divinely inspired, because I instantly responded: "You mean if I can make an Artificial Intelligence, it proves your religion is false?"
He said, "What?"
I said, "Well, if your religion predicts that I can't possibly make an Artificial Intelligence, then, if I make an Artificial Intelligence, it means your religion is false. Either your religion allows that it might be possible for me to build an AI; or, if I build an AI, that disproves your religion."
There was a pause, as the one realized he had just made his hypothesis vulnerable to falsification, and then he said, "Well, I didn't mean that you couldn't make an intelligence, just that it couldn't be emotional in the same way we are."
I said, "So if I make an Artificial Intelligence that, without being deliberately preprogrammed with any sort of script, starts talking about an emotional life that sounds like ours, that means your religion is wrong."
He said, "Well, um, I guess we may have to agree to disagree on this."
I said: "No, we can't, actually. There's a theorem of rationality called Aumann's Agreement Theorem which shows that no two rationalists can agree to disagree. If two people disagree with each other, at least one of them must be doing something wrong."
We went back and forth on this briefly. Finally, he said, "Well, I guess I was really trying to say that I don't think you can make something eternal."
I said, "Well, I don't think so either! I'm glad we were able to reach agreement on this, as Aumann's Agreement Theorem requires." I stretched out my hand, and he shook it, and then he wandered away.
A woman who had stood nearby, listening to the conversation, said to me gravely, "That was beautiful."
"Thank you very much," I said.
Part of the sequence Mysterious Answers to Mysterious Questions
Next post: "Professing and Cheering"
Previous post: "Belief in Belief"