Relatively new to the forum and just watched the 2 1/2 hour Yudkowsky video on Google. Excellent talk that really helped frame some of the posts here for me, though the audience questions were generally a distraction.
My biggest disappointment was the one question that popped up in my mind while watching and was actually posed wasn't answered because it would take about 5 minutes. The man who asked was told to pose it again at the end of the talk, but did not.
This was the question about the friendly AI: "Why are you assuming it knows the outcome of its modifications?"
Any pointer to the answer would be much appreciated.
The Soviet new "man" that Stalin wanted to create was a half-ape, half-man super-warrior.
See http://news.scotsman.com/ViewArticle.aspx?articleid=2688011
This entry reminded me of Blank Slate Asymmetry from Gene Expression. A lot of people would say the difference in our perceptions/opinions result from our general attitude toward progress, but I would suggest that it was contingent on our opposition in war to the Nazis while many of our elites were rather friendly towards the Soviets.
The Soviets weren't what I'd call transhumanists, because their New Man wasn't a definable goal or factual trend, he was a utopian catch-all of projected virtue. A transhumanist will be able to break down his goals ("uploading") into subgoals ("AI and brain scans") and roughly sketch a research path ("symbolic AI") that would either approach the goal, or fail in an informative way ("combinatorial explosion"). The Soviets could do no such thing, because NSM was nothing definable. He would certainly pop up as a consequ...
It has nothing to do with poverty of imagination and everything to do with black propaganda. The Soviets were simply never evil enough. And we know that looking forward into the future is evil, therefore the Nazis must have been guilty of that crime. If the Soviets had done it, why it may even have rehabilitated that concept. Can't have that, can we?
The problem isn't that Westerners can't imagine themselves in the shoes of the Romantic Nazis. All to the contrary, the problem is that elite conservative Westerners find it ALL TOO EASY to imagine themselves i...
It is often forgotten in the early days of proto-Nazi racial theory the Prussians were said to be the Master Race because they were a combination of German and Slav! Their combination was supposed to be just right from the perspective of Prussians, reminding me of Charles Murray's "Who wants to be an elephant?". Nietzsche also proposed breeding ubermenschen by giving Prussian officers jewish brides (haven't read him myself, just heard he said this in BG&E).
The example of Communism shows that being future-oriented will not always eliminate the "Guardians of Truth" syndrome. Sometimes it will produce people who guard a specific view of the future.
It says something about the degree to which the concept of progress permeates Western civilization, that the one is told about Nazi eugenics and hears "They tried to breed a superhuman."
What interests me is the frequent opposition to transhumanism because of transhumanism's supposedly mistaken notion of progress. Just because progress might not be smooth, it doesn't mean that we haven't experienced it in various dimensions. Skeptics about progress seem to have a romanticized view of the past, going along with a quasi-religious notion of a fall fr...
"Sometimes it will produce people who guard a specific view of the future."
Anyone read Joseph's post (just above) and immediately think 'Singularitarians!'?
Certainly the majority of (though not all) people I've met who assign that word to themselves are closer to being Guardians than Seekers. Fairly natural in that all causes want to be cults, but still likely to be harmful to the cause.
I have not noticed that, Ben.
Not all of us who believe physics-since-1600 and biology-since-1860 have seen unequivocal progress believe there has been unequivocal progress in popular political or moral opinion. The civil-rights movement of the 1950s and 1960s for example clearly represents an increase in the consistency of the application of the ideal of equality, but it constitutes unequivocal progress only if you believe that the spread of the ideal of equality constitutes unequivocal progress.
'Progress' is the accumulation of changes towards a pre-defined goal.
Just a few lifespans ago, 'progress' consisted of spreading settlers into sparsely-populated land, killing or driving away the aborigines, draining the wetlands, slaughtering the predators, and converting the ecology into farmland. It was using antibiotics widely and prophylactically, replacing ancient crop strains with monocultures, and designing our communities around the automobile.
'Progress' is the hobgoblin trotted out by everyone who thinks they know what the future should be. Anyone foolish enough to name their political goals 'progressive' ought to be excluded from the political process.
"Sometimes it will produce people who guard a specific view of the future."
Anyone read Joseph's post (just above) and immediately think 'Singularitarians!'?
Nope and I challenge you to name two examples.
"Anyone read Joseph's post (just above) and immediately think 'Singularitarians!'?"
Please report to the nearest termination center.
A Truth-Guardian is someone who 'guards' an Idea by zapping (in its myriad forms) rather than through rational argument.
Are you willing to tell me that you've never met a Singularitarian who has attacked an opponent's authority (zap), or denigrated another's work (zap), or sought to work on their Idea's strong points to the neglect of its weak points (subtle zap), or acted in an elitist manner in order to confer perceived authority on themself (smug zap), or presented new data in such a way as to strengthen their previous predictions (super Bottom Line zap...
"The moment anyone makes a biased argument because of their attachment to an Idea, they become a Guardian."
I think it's more important what happens when the bias is discovered. Does the group in question reward it or try to eliminate it? For example there is corruption in democracies as well as less free forms of government, what makes the difference is what happens when it is discovered.
Ben, of course no one is 100% un-Guardian-like, but you seemed to be claiming Singularitarians were unusually Guardian-like.
Wouldn't that make them "bio-reactionaries" or "bio-romantics"? Or has the equation of "conservatism" (which once denoted an inclination to preserve the status quo) with "reactionism" (desire to re-instate the status quo ante), "romanticism" (promotion of some vanished, idealized past), or raw fascism (power is its own logic) pervaded even these hallowed halls? Do we have a name for what was once called conservatism, or does the concept no longer have any meaningful referent?
Caledonian a Singularitarian? I doubt he knows what the word means. I don't recall him on the Singularity Institute donors list, or any of the mailing lists or websites. The term denotes activism, not belief - an "environmentalist" is not someone who believes in the existence of the environment.
Ben Jones, if the standard confirmation/disconfirmation bias is regarded as "Guardianship" then the guardian/discoverer distinction loses all meaning even with respect to scientists versus the Inquisition. The question is whether people exhib...
"an "environmentalist" is not someone who believes in the existence of the environment." Non sequitur. An environmentalist is someone who believes in the value of the environment. sloppy, sloppy, sloppy.......
Really, Chris. So if I believe in the value of the environment, but believe that it's much less valuable than the use to be gained by paving it over with strip mines, then I'm an "environmentalist"?
In any case it's a moot point. Mark Plus coined the term "Singularitarian", but didn't do much with it; when I decided to build a Singularitarian movement, I asked Mark Plus for ownership of the word and was granted it; and I define the term to involve activism. If you mean something else by the word, feel free to call yourself a "Singularian" or something.
Were/Are you joking? Seriously. I don't understand how one can own a word. Did I miss something?
I'm not disagreeing that it might involve activism (though I would define activism quite broadly), but how can one "own" a word?
Yudkowsky, I was using the colloquial meaning of the word value, that is, positive value. If you insist, positive value of a healthy environment to promote the interests of, and as defined by, the entity that assesses the value. OK ? No prob for 'ownership' of the label, my issue was with the metaphor. BTW, as I respect the issues raised here, and the expertise of those who raise them, I'd love to see a post on the biases around the concept 'ownership'.
Caledonian a Singularitarian? I doubt he knows what the word means. I don't recall him on the Singularity Institute donors list, or any of the mailing lists or websites.
It's wonderful using words in arguments when you get to redefine them. Who's the source for the alternate definition, the one that replaced "one who believes the concept of a Singularity" and the one more complex than "activist for the Singularity"? Hmmm...
I also love that you equate "working toward bringing about the Singularity" with donating money t...
I missed this the first time through:
I asked Mark Plus for ownership of the word and was granted it
I... wow. I don't quite know how to respond to a person who makes a statement such as this.
Seekers of truth do not attempt to hardwire goals and evaluations means into entities they create, whether deistic or merely offspring. Only Guardians value their beliefs so much that they attempt to transmit them as arbitrary, received 'wisdom'.
Values are not "beliefs", "true", or "false". (What about this is so hard to understand?)
Values (that is, goals of optimizers) are vastly meaningful; they affect the future shape of the universe.
A fair point, Eliezer. I'd agree that if it weren't for dis/confirmation biases, nothing would ever get done. If Einstein, when questioned about what he would have done if his special theory was disproved, had said 'meh, I can take it or leave it,' he probably wouldn't have had the drive to discover it in the first place. Attachment to your Big Idea is often what drives us.
That said, I don't see that a Big Idea About The Future is so different from a Big Idea About The Past in terms of value for humanity. Both can be open or closed, pacifistic or violent, ...
Caledonian - I'd say that one of the key concepts in my current understanding of the Singularity is that it's the polar opposite of a hard-wired goal. Surely the very idea is that we don't know what happens inside/beyond a singularity, hence the name?
The whole point of attempting a "Friendly AI" is that its proponents believe that it IS possible to exclude entire branches of possibility from an AI's courses of action - that the superhuman intelligence can be made safe. Not merely friendly in a human sense, but favorable to human interests, not...
caledonian said: "Perhaps the possibility that a consequence of an entity being utterly good might be its being utterly unsafe has never occurred to them."
This describes monotheism rather well. It has occured to me.
Caledonian,
Yes, it has occurred to 'them'. I hope you haven't read http://www.singinst.org/AIRisk.pdf, since if you have, you haven't grasped the challenge. The crux isn't excluding branches of possible action by an AI, it's ensuring those avenues aren't attractive options for any reason.
The crux isn't excluding branches of possible action by an AI, it's ensuring those avenues aren't attractive options for any reason.
(agog)
Would you care to explain what the distinction between those two states is?
Sure - it's the difference between not stealing because you think you'll get caught and go to prison, and not stealing because you think theft is irrational/immoral/wrong/you name it. The first is sociopathy, the second is what we'd term normal human reasoning. Can I assume you believe there is no such thing as a Friendly AI?
When you're determining the value structure of a mind, "ensuring those avenues aren't attractive options for any reason" IS excluding them from the set of possible courses of action. The key phrase there is for any reason.
As for the rest of your argument, reasoning is precisely what the normal human does NOT do, and it's hilarious that you think logical arguments are what keeps most people from theft.
Caledonian, shouldn't you check up on who currently owns the word 'reasoning' before stating that?
I guess there must be some sort of register somewhere...
A minor semantic point: wouldn't advocating a return to the ancient Nordic race make them racial reactionaries rather than racial conservatives?
Taking the British National Party as an example of a racial conservative group, we see that they endeavour to PRESERVE the white race. They believe the master race (or, in this case, the race that somehow deserves ownership of the UK) is extant, and must be protected. On the other hand, the Nazi wished to RESTORE a racial standard that they believed had been long buried.
This is somewhat true. (It gets even stranger when you find out that they were also trying to similar things with animals, trying to somehow breed dogs back to the first dog ancestor.) However, it's worth noting that Nazis directly tapped into the common "eugenics" mode of thought in our society, and eugenicists in general were trying to "breed better humans" (by doing things like encouraging the forced sterilization of the insane and the physically disabled, ect).
Of course, it's still a fundamental fail of an idea all the way around...
"if you failed hard enough to endorse coercive eugenics"
This might be found a bit too controversial, but I was tempted to come up with not-so-revolting coercive eugenics system. Of course it's not needed, if there is technology for correcting genes, but let's say we only have circa 1900 technology. It has nothing to do with the point of Elizer's note, it's ust my musing.
Coervie eugenics isn't strictly immoral itself. It is a way of protecting people not yet born from genetical flaws - possible diseases, etc. But even giving them less then optimal...
Like any educated denizen of the 21st century, you may have heard of World War II. You may remember that Hitler and the Nazis planned to carry forward a romanticized process of evolution, to breed a new master race, supermen, stronger and smarter than anything that had existed before.
Actually this is a common misconception. Hitler believed that the Aryan superman had previously existed—the Nordic stereotype, the blond blue-eyed beast of prey—but had been polluted by mingling with impure races. There had been a racial Fall from Grace.
It says something about the degree to which the concept of progress permeates Western civilization, that the one is told about Nazi eugenics and hears "They tried to breed a superhuman." You, dear reader—if you failed hard enough to endorse coercive eugenics, you would try to create a superhuman. Because you locate your ideals in your future, not in your past. Because you are creative. The thought of breeding back to some Nordic archetype from a thousand years earlier would not even occur to you as a possibility—what, just the Vikings? That's all? If you failed hard enough to kill, you would damn well try to reach heights never before reached, or what a waste it would all be, eh? Well, that's one reason you're not a Nazi, dear reader.
It says something about how difficult it is for the relatively healthy to envision themselves in the shoes of the relatively sick, that we are told of the Nazis, and distort the tale to make them defective transhumanists.
It's the Communists who were the defective transhumanists. "New Soviet Man" and all that. The Nazis were quite definitely the bioconservatives of the tale.