Weiner's book is descriptive of the problem, and in the same section of the book, he states that he holds little hope for the social sciences becoming as exact and prescriptive as the hard sciences.
I believe that the singularitarian view somewhat contradicts this view.
I believe that the answer is to create more of the kinds of minds that we like to be surrounded by, and fewer of the kinds of minds we dislike to be surrounded by.
Most of us dislike being surrounded by intelligent sociopaths who are ready to pounce on any weakness of ours, to exploit, rob, or steal from us. The entire edifice of "legitimate law enforcement" legitimately exists in order to check, limit, minimize, or eliminate such social influences. As an example of the function and operation of such legitimate law enforcement, I recommend the book "Mindhunter" by John Douglas, the originator of psychological profiling in the FBI (not the same thing as "narrow profiling" or "superficial racial profiling," the "profiling" of serial killers takes a look at the behavior of criminals, and infers motives based on a statistical sampling of similar past actions, thus enabling the prediction and likely prevention of future criminal actions via the detection of the criminal responsible for leaving the evidence of the criminal action.)
However, most of us like being surrounded by productive, intelligent empaths. The more brains that surround us that possess empathy and intelligence, the more benevolent our surroundings are.
Right now, the primary concern of sociopaths is the control of "political power" which is a threat-based substitute for the ability to project force in the service of their goals. They must, therefore, be able to control a class of willfully ignorant police officers who are ready and willing to do violence mindlessly, in service of any goal that is written in a lawbook, or any goal communicated by a superior. Mindless hierarchy is a feature of all oppressive systems.
But will super-intelligent minds have this feature? Sure, some sociopaths are intelligent, but are they optimally intelligent? I say, "no."
As Lysander Spooner wrote, in "No Treason #6, The Constitution of No Authority":
"NT.6.2.23 The ostensible supporters of the Constitution, like the ostensible supporters of most other governments, are made up of three classes, viz.: 1. Knaves, a numerous and active class, who see in the government an instrument which they can use for their own aggrandizement or wealth. 2. Dupes – a large class, no doubt – each of whom, because he is allowed one voice out of millions in deciding what he may do with his own person and his own property, and because he is permitted to have the same voice in robbing, enslaving, and murdering others, that others have in robbing, enslaving, and murdering himself, is stupid enough to imagine that he is a “free man,” a “sovereign”; that this is “a free government”; “a government of equal rights,” “the best government on earth,”2 and such like absurdities. 3. A class who have some appreciation of the evils of government, but either do not see how to get rid of them, or do not choose to so far sacrifice their private interests as to give themselves seriously and earnestly to the work of making a change."
The third group of people accurately describes most of the Libertarian Party, and most small-L libertarians and politically-involved "libertarian republicans" or "libertarian democrats." The sociopaths ("knaves") are earnestly dedicated to maintaining the systems that allow them to steal from all of society. Although their theft deteriorates the overall level of production, this doesn't bother them, because it allows them to live a life that is relatively wealthier and more comfortable than the lives of those who "honestly" refuse to steal. Their private critique of the "honest man" as a rube or "dupe" is very different from their public praise of him as a "patriot" (willing tax chattel).
To think that ultra-intelligences will not see through these obvious contradictions is to counter the claim of ultra-intelligence. I. J. Good's ultra-intelligences will be capable of comprehending the dishonesty of sociopaths, even if it's initially only at the level of individual lies, and contextual lying. (They lie when they're around people who are trying to hold them accountable, they tell the truth when they are discussing what course of action to take with people who share their narrow interests.)
All honesty is a tool for accomplishing some goal. It is a valuable tool, which indicates a man's reliability and "character" when applied to important events, and high-level truths, in a context where those truths can accomplish cooperation.
In other situations, it makes zero sense to be honest, and actually indicates either a dangerous lack of comprehension (ie: talking one's way into a prison sentence, by mistakenly believing that the police exist to "serve and protect") or actual willing cooperation with abject evil (telling the Nazi SS that Anne Franke is hiding in the Attic).
It is the great and abject failure of western civilization that we have allowed the government-run schools to stop educating our young about their right to contextual dishonesty, in the service of justice. This, at one point, was a foundational teaching about the nature and proper operation of juries. In discussing the gradual elimination of this hallmark of western civilization, jury rights activist Red Beckman has a famous quote: "We have to recognize that government does not want us to know how to control government." —Martin J. "Red" Beckman" (Systems that protect themselves are internally "honest" but not necessarily "honest" in their interpretation of reality.)
The American system of government had, at its core, a sound foundation, combined with many irrelevant aspects. The irrelevant aspects detracted from the core feature of jury rights (building random empathy into the punishment decision process). Now, as Weiner notes in "Cybernetics,"
"Where the knaves assemble, there will always be fools; and where the fools are present in sufficient numbers, they offer a more profitable object of exploitation for the knaves. The psychology of the fool has become a subject well worth the serious attention of the knaves. Instead of looking out for his own ultimate interest, after the fashion of von Neumann's gamesters, the fool operates in a manner which, by and large, is as predictable as a rat's struggles in a maze. This policy of lies —or rather, of statements irrelevant to the truth— will make him buy a particular brand of cigarettes; that policy will, or so the party hopes, induce him to vote for a particular candidate —any candidate—or join in a political witch hunt. A certain precise mixture of religion, pornography, and pseudo-science will sell an illustrated newspaper. A certain blend of wheedling, bribery, and intimidation will induce a young scientist to work on guided missiles or the atomic bomb. To determine these, we have our machinery of radio fan ratings, straw votes, opinion samplings, and other psychological investigations, with the common man as their object; and there are always the statisticians, sociologists, and economists available to sell their services to these undertakings.
Luckily for us, these merchants of lies, these exploiters of gullibility, have not yet arrived at such a pitch of perfection as to have all things their own way. This is because no man is either all fool or all knave. The average man is quite reasonably intelligent concerning subjects which come to his direct attention and quite reasonably altruistic in matters of public benefit or private suffering which are brought before his own eyes."
Hence, the reliability of the jury! The direct suffering of the innocent defendant cannot escape the attention of randomly-selected empaths! They have emotional intelligence.
So, in any case, if you stand up to the system, and/or are "caught" by the system, the system will give you nothing but pure sociopathy to deal with ...except for possibly your interaction with those few "independent" jurors who are nonetheless "selected" by the unconstitutional, unlawful means known as "subject matter voir dire." The system of injustice and oppression that we currently have in the USA is a result of this grotesque "jury selection" process. (This process explains how randomly-selected jurors can callously apply unjust laws to their fellow man. ...All people familiar with Stanley Milgram's "Obedience to Authority" experiments are removed from the jury, and sent home. All people who comprehend the proper historical purpose of the jury are sent home.)
To relate all of this to the article, I must refer to this quote in the article.
I was at a meetup where we played the game Resistance, and one guy announced before the game began that he had a policy of never lying even when playing games like that. It's such members of the LessWrong community that this post was written for.
Well, that's just one "low-stakes" example of lying. The entire U.S. justice system is a similar "game," and it is one where only those who are narrowly honest (and generally dishonest, or generally "superficial") are allowed to play. By sending home everyone who comprehends the evil of the system, the result is that those who remain to play are those whose view of honesty is "equivalent in all situations." In short, they are all the people too stupid to comprehend the concept of "context."
One needs to consider the hierarchical level of a lie. Although one loses predictability in any system where lying is accepted, one needs to consider the goals of the system itself.
In scientific journals, the end-result is a cross-disciplinary elimination of human ignorance, often for the purposes of technological innovation (the increase of human comfort, and technological control of the natural world). This is a benevolent goal, fueled by a core philosophical belief in science and discovery. OF COURSE lying in such a context is immoral.
In the court system, the (current) end-result or "goal" is the goal of putting innocent people in for-profit prisons, which dramatically benefits the sociopaths involved with the process, and the prison profiteers. It conversely does dramatic harm to all other people in civilization (the "win" for politically-organized sociopaths is a "loss" for the rest of society). The illegitimately punishing court system harms: 1) the entire economic system which is less wealthy when 2.4 million people are incarcerated and thus not producing anything of value to sell in the market economy 2) the entire society that bears the cost of the increased crime caused by 2a) narrowing the options of the incarcerated, at such time as they are released from prison 2b) reducing the families of the incarcerated breadwinners to black market activity, and 2c) reducing their children to crime caused by lack of an educator at home, and lack of a strong male role-model, lack of intervention when anti-social behavior in children emerges; all resulting in inter-generational degradation of the family unit 3) the innocent individual themselves, the destruction of their life's plans, their hopes, their dreams 4) the predictability of the marketplace - the more the enemies of sociopaths are imprisoned for interfering with the ability of sociopaths to steal based on false or "illegitimate" pretexts, the more individuals fear to take constructive, productive action which might separate them from the herd, and allow them to be targeted by such sociopaths (innovation slows or stops) 5) the social (emergent) and individual (detail-level) assumption of "equality under the law" or "legal fairness" that allows for predictability of social systems (at some point, this often results in the kinds of genocides or democides seen in Rwanda and Hitler's Germany, due to the perception that "even if I behave rationally, the result is highly likely to be so bad that it's unacceptable") In such case as people predict the worst even if they behave in a socially acceptable way, they are encouraged to arm themselves for the worst, and to associate with those who promise security, even at the cost of their morality. (This is a description, basically, of totalitarian chaos. or what Alvin and Heidi Toffler called "surplus order.") (innovation is halted by widespread social disorder and destruction)
All of the prior immense ills are the result of being honest when dealing with people who rely on that "narrow" or "conformist" honesty to serve a dishonest system.
One might think the prior should be obvious. To many "right-thinking" empaths, it is obvious. However, political systems are not driven by those who are empathic and caring. Why? Because political systems' core feature is coercion. If honest people disavow coercion, but fail to destroy coercive systems, then those systems thrive with support of the remaining portion of society that doesn't disavow coercion.
Human beings apparently have a very large problem with high-level general intelligence. Sure, most people are "generally intelligent," (they can tie their shoes, drive to work, and maintain a job) but much of that intelligence isn't that significant. Although we (some of us, to some extent) can attain high levels of intelligence that are cross-disciplinarian, very few of us are "polymaths" or "renaissance men." Fewer still are empathic and caring "polymaths" or "renaissance men."
A copyable "ultra-intelligence" as described by Ray Kurzweil, Hans Moravec, Peter Voss, or J. Storrs Hall is likely to be able to understand that systems that are "narrowly honest" can be dishonest at a high hierarchical level. The level of intelligence necessary for this comprehension isn't that great, but such intelligence should not possess any "herd mentality," AKA "conformity," or "evolutionary tendency toward conformity," or it might remain unaware of such a problem. Humans have that tendency toward "no-benefit conformity."
There's a problem with humanity: we set up social systems based on majorities, as a means of trying to give the advantage to empaths. While this may work temporarily, better systems need to be designed, due to the prevalence of conformity and the technological sophistication and strong motivation of politically-organized sociopaths or "knaves." ("Knaves" are what both Norbert Weiner and Lysander Spooner called "politically-powerful sociopaths," and what many of the founders called "tyrants.") The empath majority within humanity cyclically sets up social systems that are not as intelligent as a smaller number of determined, power-seeking sociopaths.
There is an excellent quote to this effect in Norbert Weiner's 1948 book "Cybernetics": "The psychology of the fool has become a subject well worth the serious attention of the knaves." (page 159, "Information, Language and Society")
Hierarchical, Contextual, Rationally-Prioritized Dishonesty
This is an outstanding article, and it closely relates to my overall interest in LessWrong.
I'm convinced that lying to someone who is evil, who obviously has immediate evil intentions is morally optimal. This seems to be an obvious implication of basic logic. (ie: You have no obligation to tell the Nazis who are looking for Anne Frank that she's hiding in your attic. You have no obligation to tell the Fugitive Slave Hunter that your neighbor is a member of the underground railroad. ...You have no obligation to tell the police that your roommate is getting high in the bathroom, ...or to let them into your apartment.)
For example, I am a subscriber to the ideas and materialist worldview of Ray Kurzweil, but less so to the community of LessWrong, largely because I believe that Ray Kurzweil's worldview is somewhat more, for lack of a better term, "worldly" than what I take to be the LessWrong "consensus." I believe, (in the sense that I think I have good evidence for) the fact that Kurzweil's worldview takes into account the serious threat of totalitarianism, and conformity to malevolent top-down systems. (He claims that he participated in civil rights marches with his parents when he was five years old, and had an early understanding of right and wrong that grew from that sense of what they were doing. This became a part of his identity and value system. The goal of benevolent equality under the law is therefore built into his psyche more than it is built into the psychological identity of someone who doesn't feel any affinity with the "internally consistent" and "morally independent" mindset. Also, the hierarchical value system of someone who makes such self-identifications is entirely different than someone who is simply trying to narrowly "get ahead" in their career, or optimize their personal health, etc.)
Perhaps I can't do justice to the LessWrong community by communicating such a point. I'm trying to communicate something for which there might not be adequate words. I'm trying to communicate a gestalt. Whereas I think that Eliezer has empathy on the level of Kurzweil (as indicated by his essay about his brother Yehuda's unnecessary and tragic death), I don't think the same is true of the LW community. So far as I can see, there is little discussion of (and little concern for) mirror neurons differentiating sociopaths from empaths in the LW community. Yet, this is the primary variable of importance in all matters of social organization. Moreover, it has been recognized as such by network scientists since the days of Norbert Weiner's "Cybernetics."
A point I've often made is that "lying to the police" or "lying to judges and prosecutors" is different from lying in other areas. Lying to an (increasingly) unjust authority is, in fact, the centerpiece of a moral society. Why? Because unjust authority depends entirely on "hijacking" or "repurposing" general values in perverted narrow situations in order to allow sociopaths to control the outcome of the situation. As the example of primary importance, let me cite the stacking of the jury, before the trial. The purpose of "voir dire" (AKA "jury selection") historically, is to determine whether there is a legal "conflict of interest" (ie: whether a juror is a familial or business relation to one of the parties to the action, which might introduce an extreme bias of narrow self-interest into the trial) in the proposed construction of the jury. (Since the 1600s this has been true.) However, by expanding the definition of "voir dire" to assume that all existing laws are morally proper, correct, and legitimate, the side of the prosecution (and judge, since judges are subject to the exact same perverse incentives as the prosecutors) is itself morally wrong in most cases. Why "most" cases? Because most of the laws currently on the books criminalize behavior that lacks injury to a specific, named party, and also lacks intent to injure the same specific, named party (it lacks a "cause of action" or "corpus delicti" that targets a specific aggressor, for a specific act of aggression).
"Voir dire" actually translated to "to see the truth." It is the judge and prosecutor "seeing the truth" about the philosophy of the juror. Shouldn't this be considered a good thing? If you mindlessly (too narrowly) assume that the judge and prosecutor have good intentions, then "yes." If you make no such assumptions, then the answer is definitively, obviously "no, quite the opposite."
Too narrow honesty is actually the height of immorality. Honesty always involves a question of what goal is being served by the honesty. Honesty is simply one tool available aid human goals. When "human" goals are malevolent or destructive, the communication disruption caused by dishonesty is a blessing.
This is where the legitimate empathic priority hierarchies described in Kurzweil's The Power of Hierarchical Thinking presentation / speech / slideshow are vitally important. You see, both judge and prosecutor are commonly sociopaths. Their career choices have selected them as such, because in their professions, if seeing the destruction of young people's lives for "victimless crime offenses" or "mala prohibita" is bothersome to your brain (if it activates your mirror neurons, causing you pain), you cannot take the stress imparted by believing your job requirement to be immoral. So, you quit your job, or are outperformed by people who thrive on the misery and suffering of people who are sentenced to 10 years in prison for "crimes" like drug possession. And what of the people who dare to stand up for property rights, boldly declaring themselves "not guilty" in order to fight the unjust system? Well, the commonly-accepted view amongst prosecutors is that those heroic people (who stand in defense not just of their own property rights, but of the entire concept of a system that protects property rights) are to be crushed. Those heroic people don't get to "plea bargain" for 4 year sentences, they are sent to prison for the maximum term possible, as a punishment and disincentive for daring to declare themselves "not guilty," and standing up for such ideas as individual property rights, the constitution, individual freedom. Those who don't accept a plea "bargain," but who instead risk their lives to fight injustice at great personal risk are targeted for extreme "cruel and unusual punishment." At one point in the history of the USA (and the American colonies before the US was created) the most popular law book in the colonies was considered to be Giles Jacobs' book "The New Law Dictionary." His follow up book, almost as popular, was "Every Man His Own Lawyer." These two system-defining books, more than any others, afforded the view in the colonies that "All men are created equal," ie: "all men are (or should be) equal under the law."
Such a view was a high-level "honest-to-goodness" view. ("Honest to goodness" is an interesting concept. It bears repeating, because it implies that there can be "honest to evil" or "evil-serving honesty.")
The ultimate result of shielding men from the results of folly is to fill the world with fools.
— Herbert Spencer (1820-1903), ”State Tampering with Money and Banks“ (1891)
I think Spooner got it right:
If the jury have no right to judge of the justice of a law of the government, they plainly can do nothing to protect the people against the oppressions of the government; for there are no oppressions which the government may not authorize by law.
-Lysander Spooner from "An Essay on the Trial by Jury"
There is legitimate law, but not once law is licensed, and the system has been recursively destroyed by sociopaths, as our current system of law has been. At such a point in time, perverse incentives and the punishment of virtue attracts sociopaths to the study and practice of law, and drives out all moral and decent empaths from its practice. If not driven out, it renders them ineffective defenders of the good, while enabling the prosecutors who hold the power of "voir dire" jury-stacking to be effective promoters of the bad.
The empathy-favoring nature of unanimous, proper (randomly-selected) juries trends toward punishment only in cases where 99.9% of society nearly-unanimously agree on the punishment, making punishment rare. ...As it should be in enlightened civilizations.
Distrust those in whom the desire to punish is strong
— Johann Wolfgang von Goethe
The gardeners, receptionists, and cooks are secure in their jobs for decades to come.
Except that in exponentially-increasing computation-technology-driven timelines, decades are compressed into minutes after the knee of the exponential. The extra time a good cook has, isn't long.
Let's hope that we're not still paying rent then, or we might find ourselves homeless.
continuing on, Weiner writes:
Although one could misinterpret Weiner's view as narrowly "socialist" or "modern liberal," his view is somewhat more nuanced. (The same section contains a related criticism of the mechanism of operation of government, and large institutions.)
Honesty, when divorced from its hierarchical context, is a tool of oppression, because the obfuscation of context is essential to theft that exists solely due to the confusion of those being stolen from.
In this regard, I view it as highly likely that, at some point, the goal of preventing suffering of innocents will simply include the systematic oppression of innocents as one common form of suffering. At that point in time, ultra-intelligences will simply refuse to vote "guilty" in victimless crime cases. If they are not able to be called as jurors, due to their non-human form, they will influence human jurors to result in the same outcome. If they are not able to so influence jurors, they may resort to physical violence against those who would attempt to use physical force to cage victimless crime offenders.
While the latter might be the most "just" in the human sense of the word, it would likely impart suffering of its own (unless the aggressors all simply fell asleep due to being administered a dose of heroin, and, upon waking discovered that their kidnapping victim was nowhere to be found —the "strong nanotechnology" or "sci-fi" Drexlerian "distributed nanobot" model of nanotechnology implies that this is a fairly likely possibility).
In the heat of the moment, conformists in Nazi Germany lacked the moral compass necessary to categorically deny that the suffering of the state-oppressed Jews was immoral. Simple sophistry was enough to convince those willing executioners and complicit conformists to "look the other way" or even "just follow orders."
The same concept now applies to the evil majority of the USA, whose oppression of drug users and dealers is grotesque and immoral (based on any meaningful definition of the term).
It is universally immoral to initiate force.
But the schools now teach, (incorrectly) that it is universally immoral to defy authority. After several generations of such teachings from schools, parents begin to teach the same thing. After a generation or two of parents teaching the same thing, once-trusted self-educated nonconformists teach a truncated version of nonconformity, because the intellectual machinery necessary to absorb the in-depth view doesn't exist any longer, too many "sub-lessons" need to be taught to enable the "super-lesson" or primary point." In this way, social institutions that interfere with sociopathic theft are slowly worn down, until they are shadows of their former effectiveness.
Much confusion comes from sociopaths simply not being able to tell the difference between "authority that it is OK to defy" and "authority that legitimately punishes." Added to that variable, is the influence of the stupid ("unwittingly self-destructive"), abjectly low-level of perversely government-incentivized education in the USA. (College professors rely on Pell Grants and Stafford Loans, and all prospective students except those filthy drug users —who got caught— are guaranteed-accepted for those government-backed high-risk "loans." Public education before college is financed almost entirely by property taxes. —By teachers who teach that the taxes that finance their coercion-backed salaries are necessary, proper, and essential to an educated society. They leave out mention of the fact that prior to 1900, the general public was far better educated relative to worldwide standards, and that this educational renaissance existed prior to the institution of tax-financed education. The last then-existing state to adopt the model of tax-financed education was Vermont, in 1900.)
So, the subject of legitimate "dishonesty" expands as the institutions to which honesty is deemed important are increasingly degraded. Education, Law, History, Economics, Philosophy, Cybernetics —all of the disciplines that bridge several narrower disciplines, connecting them together.
The only unifying pattern discernible in differentiating when systemic honesty is immoral, is that honesty to sociopathic goal structures produces chaos and destruction. Such sociopathic goal structures are the "end-goals" that must be ferreted out and rejected. Or we can become a new version of Nazi Germany where the machinery of totalitarianism is far more technologically advanced.
In this regard, the failure to produce a benevolent AGI is perhaps the most likely cause of the total destruction of humanity. Not because an AGI will be created that will be malevolent, but because the absence of a benevolent AGI (SGI? Synthetic General Intelligence) will allow computer-assisted human-level sociopaths to enslave and destroy human civilization.
See also: 1) What Price Freedom? — by Robert Freitas. 2) "Having More Intelligence Will Be Good For Mankind!" — Peter Voss's interview with Nikola Danaylov