I don't know what document that link originally pointed to, but this document contains one of Jaynes's earliest (if not the earliest) descriptions of the idea.
Stephen R. Diamond, there are two distinct things in play here: (i) an assessment of the plausibility of certain statements conditional on some background knowledge; and (ii) the relative frequency of outcomes of trials in a counterfactual world in which the number of trials is very large. You've declared that probability can't be (i) because it's (ii) -- actually, the Kolmogorov axioms apply to both. Justification for using the word "probability" to refer to things of type (i) can be found in the first two chapters of this book. I personally cal... (read more)
Vilhelm S., companies and people who lose money on CDOs have mortgages to pay and employ people who have mortgages to pay... Once the system gets coupled like that, one unlucky bet can start the cascade. I'm not saying this actually happened, but it's a mechanism which could falsify the assertion "the lack of correlation doesn't stop being real just because people believe in it".
David, the inelegance is that the study asked adults in general to imagine parental grief rather than asking parents in particular. (Your correct observations about imagined versus actual grief were already set forth in the post.)
This post helps to ease much of what I have found frustrating in the task of understanding the implications of EP.
Huh. I guess I just don't see Angel (the TV character, not the commenter) as the equivalent of the verthandi. (Also naming the idea after the actor instead of the character lead me somewhat astray.)
If you google boreana you should get an idea of where that term comes from, same as verthandi.
Still need a little help. Top hits appear to be David Boreanaz, a plant in the Rue family, and a moth.
No. I asserted that...
This might be a good idea... At this point, the "hedonic impact" of this mechanic will almost disappear.
I don't disagree with this. My scenario is premised on the reward being a surprise, so it implicitly assumes one-time use, or at least no overuse.
Well, that is even worse, because essentially, you just took the choice away from player.
I can't help but feel that you didn't really bother to think this response through. Taken literally, you've just asserted that a surprising reward with character synergy is worse than a surprising rigid reward that makes the player feel regret. You assert that this is so because choice was taken away from the player even though neither situation involves player choice.
I get that yout design principle is to give the player choice and the ability to plan. So what is the right way to give "good news" to the player with the most hedonic impact?
The emphasis on Bayesian probability is because it is the simplest way to extend classical logic to propositions with varying degrees of plausibility. Just as all classical logic can be reduced to repeated applications of modus ponens, all manipulations of plausibility can be reduced to applications of Bayes' Theorem (assuming you want results that will line up with classical logic as the plausibilities approach TRUE and FALSE).
If some or all abilities are hidden at the beginning, that forces the player to choose based on incomplete knowledge, and more often that not, leads to regrets: "I wish I purchased that ability which turned out to work in nice synergy with others, and not this one which turned out to be useless..". Especially if there's some finite pool of resources used to purchase these abilities.
And that is not fun, even if surpising.
This seems to miss the point -- you're talking about a surprise that isn't a pleasant surprise. Suppose the game was designed... (read more)
Utopia originally meant no-place, I have a hard time forgetting that meaning when people talk about them.
The term "utopia" was a deliberate pun on "outopia" meaning "no place" and "eutopia" meaning "good place". It seems doubtful that Thomas More actually intended to depict his personal ideal society, so one might say that Utopia is the original Weirdtopia.
I think we're looking at premature search-halts here.
I plead no contest.
Economic Weirdtopia: FAIth determines that the love of money actually is the root of ~75% of evil, so it's back to the barter system for us.
Sexual Weirdtopia: FAIth determines that the separatist feminists were right -- CEV requires segregation by sex. Homosexual men and lesbians laugh and laugh. Research on immersive VR becomes a preoccupation among the heterosexual majority in both segregated camps.
Not very plausible, but... "That's the thing about FAIth. If you don't have it, you can't understand it. And if you do, no explanation is necessary."
I don't yet see quantifiable arguments why from-scratch AI is easier [than human augmentation].
From-scratch AI could also be justified as yielding greater benefits even if it as difficult (or more difficult) than human augmentation.
Cyan, is that a standard hypothesis? I'm not sure how "practice" would account for a very gregarious child lacking an ordinary fear of strangers.
I don't know if it's a standard hypothesis -- it's just floating there in my brain as background knowledge sans citation. It's possible that read it in a popular science book on neuroplasticity. I'd agree that "practice" doesn't plausibly account for the lack of ordinary fear; it's intended as an explanation for the augmentations, not the deficits.
Nitpick for Doug S.: that's actually two coupled evolutionary limits. Babies' heads need to fit through the women's pelvises, which also have to be narrow enough for useful locomotion.
Deacon makes a case for some Williams Syndrome symptoms coming from a frontal cortex that is relatively too large for a human, with the result that prefrontal signals - including certain social emotions - dominate more than they should.
Having not read the book, I don't know if Deacon deals with any alternative hypotheses, but one alternative I know of is the idea that WSers get augmented verbal and social skills is because it is the only cognitive skill they are able to practice. In short, WSers are (postulated to be) geniuses at social interaction beca... (read more)
TGGP, I think it's supposed to. The General is quoted in the linked article.
Would it have been the moral thing to do to turn around and leave the Indians alone, instead of taking their land and using it to build an advancing civilization...?
If you invoke the unlimited power to create a quadrillion people, then why not a quadrillion?
One of these things is much like the other...
Jeff, if you search for my pseudonym in the comments of the "Natural Selection's Speed Limit and Complexity Bound" post, you will see that I have already brought MacKay's work to Eliezer's attention. Whatever conclusions he's come to have already factored MacKay in.
Eliezer, you'd have done better to ignore ReadABook's trash. Hir ignorance of your arguments and expertise was obvious.
Peter de Blanc, I don't have an example, just a vague memory of reading about minimax-optimal decision rules in J. O. Berger's Statistical Decision Theory and Bayesian Analysis. (That same text notes that minimax rules are Bayes rules under the assumption that your opponent is out to get you.)
IIRC, there exist minimax strategies in some games that are stochastic. There are some games in which it is in fact best to fight randomness with randomness.
For what it's worth, Tim Tyler, I'm with you. Utility scripts count as programs in my books.
I mean, we weren't even designed by a mind, we sprung from simple selection!
This is backwards, isn't it? Reverse engineering a system designed by a (human?) intelligence is a lot easier than reverse engineering an evolved system.
Emile, you've mixed up "optimization process" and "intelligence". According to your post, Eliezer wouldn't consider evolution an optimization process. He does; he doesn't consider it intelligent.
...it seems to me much of the beautiful LaTex equations and formulas are only to give the impression of rigor.
I didn't suggest equations to enforce some false notion of rigor -- I suggested them as an aid to clear communication.
Jef Allbright, it seems to me that if you want Eliezer to take your criticisms seriously, you're going to need more equations and fewer words. (It would be nice if Eliezer produced some equations too.)
"But I still suspect that there's a little distance there, that wouldn't be there otherwise, and I wish my brain would stop doing that."
A finely crafted recursion. I salute you.
So in my posts on this topic, I proceeded to (attempt to) convey a larger and more coherent context making sense of the ostensible issue.
Right! Now we're communicating. My point is that the context you want to add is tangential (or parallel...? pick your preferred geometric metaphor) to Eliezer's point. That doesn't mean it's without value, but it does mean that it fails to engage Eliezer's argument.
But it seems to me that I addressed this head-on at the beginning of my initial post, saying "Of course the ends justify the means -- to the extent that a... (read more)
Since you said you didn't know what to do with my statement, I'll add, just replace the phrase "limit the universe of discourse to" with "consider only" and see if that helps. But I think we're using the same words to talk about different things, so your original comment may not mean what I think it means, and that's why my criticism looks wrong-headed to you.
By subsequent discussion, I meant Phil Goetz's comment about Eliezer "neglecting that part accounted for by the unpredictability of the outcome". I'm with him on not understanding what "a model of evolving values increasingly coherent over increasing context, with effect over increasing scope of consequences" means; I also found your reply to me utterly incomprehensible. In fact, it's incredible to me that the same mind that could formulate that reply to me would come shuddering to a halt upon encountering the unexceptionable phrase "universe of discourse".
But in an interesting world of combinatorial explosion of indirect consequences, and worse yet, critically underspecified inputs to any such supposed moral calculations, no system of reasoning can get very far betting on longer-term specific consequences.
This point and the subsequent discussion are tangential to the point of the post, to wit, evolutionary adaptations can cause us to behave in ways that undermine our moral intentions. To see this, limit the universe of discourse to actions which have predictable effects and note that Eliezer's argument still makes strong claims about how humans should act.
1) Do you believe this is true for you, or only other people?
I don't fit the premise of the statement -- my cherished spouse is not yet late, so it's hard to say.
2) If you know that someone's cherished late spouse cheated on them, are you justified in keeping silent about the fact?
3) Are you justified in lying to prevent the other person from realizing?
4) If you suspect for yourself (but are not sure) that the cherished late spouse might have been unfaithful, do you think that you will be better off, both for the single deed, and as a... (read more)
Fact check: MDL is not Bayesian. Done properly, it doesn't even necessarily obey the likelihood principle. Key term: normalized maximum likelihood distribution.
...the language and the arts with my comparison...
I thought about going this way, but I decided to stick with what I know.
Since sarcasm seems to have failed, let me just state flatly that all of the cultures we've mentioned have enough members and enough diversity that blanket assertions such as, "Japanese martial arts are worse than Chinese ones," or "American football is a cheap knockoff of rugby" are reductive and parochial to the point of not-even-wrongness.
Most American culture seems like a reinvention of British culture demanded by national pride. My impression is that their versions are like cheap knock-offs of the originals. Their beers are worse. They even managed to mess up the game of football. America faces limitations due to their vast tracts of underpopulated flyover country. That's a problem Britain doesn't have.
For those who are interested, a fellow named Kevin Van Horne has compiled a nice unofficial errata page for PT:LOS here. (Check the acknowledgments for a familiar name.)
This is a false dichotomization. Everything is reality!
"Quotation mode" is analogous to an escape character. There's no dualism here.
"Consider the horror of America in 1800, faced with America in 2000. The abolitionists might be glad that slavery had been abolished. Others might be horrified, seeing federal law forcing upon all states a few whites' personal opinions on the philosophical question of whether blacks were people, rather than the whites in each state voting for themselves. Even most abolitionists would recoil from in disgust from interracial marriages - questioning, perhaps, if the abolition of slavery were a good idea, if this were where it led. Imagine someone from 18... (read more)
Science: Best Sans Booty.
Schrödinger disagreed. (So did Einstein... and Feynman... I could mention Kinsey, but that would be cheating, I supppose.)
Jaynes was a really smart guy, but no one can be a genius all the time. He did make at least one notable blunder in Bayesian probability theory -- a blunder he could have avoided if only he'd followed his own rules for careful probability analysis.
Eliezer, I think you have dissolved one of the most persistent and venerable mysteries: "How is it that even the smartest people can make such stupid mistakes".
Michael Shermer wrote about that in "Why People Believe Weird Things: Pseudoscience, Superstition, and Other Confusions of Our Time". In the question of smart people believing weird things, he essentially describes the same process as that Eliezer experienced: once smart people decide to believe a weird thing for whatever reason, it's much harder to to convince them that their beliefs are flawed because they are that much better at poking holes in counterarguments.
One disturbing thing about the Petrov issue that I don't think anyone mentioned last time, is that by praising nuclear non-retaliators we could be making future nuclear attacks more likely by undermining MAD.
Petrov isn't praised for being a non-retaliator. He's praised for doing good probable inference -- specifically, for recognizing that the detection of only 5 missiles pointed to malfunction, not to a U.S. first strike, and that a "retaliatory" strike would initiate a nuclear war. I'd bet counterfactually that Petrov would have retaliated if the malfunction had caused the spurious detection of a U.S. first strike with the expected hundreds of missiles.
You've got to be almost as smart as a human to recognize yourself in a mirror...
Quite recently, research has shown that the above statement may not actually be true.
Eliezer, I think you meant to say that "19 * 103 might not be 1957" instead of 1947. Either that or I'm misunderstanding that entire paragraph.
The setup's a little opaque, but I believe the correct reading is that the other person (characterized as honest) is correcting the faulty multiplication of the notional reader ("you").
Barkley Rosser, there definitely is something a little hinky going on in those infinite dimensional model spaces. I don't have the background in measure theory to really grok that stuff, so I just thank my lucky stars that other people have proven the consistency of Dirichlet process mixture models and Gaussian process models.
Barkley Rosser, what I have in mind is a reality which in principle predictable given enough information. So there is a "true" distribution -- it's conditional on information which specifies the state of the world exactly, so it's a delta function at whatever the observables actually turn out to be. Now, there exists unbounded sequences of bits which don't settle down to any particular relative frequency over the long run, and likewise, there is no guarantee that any particular sequence of observed data will lead to my posterior distribution gett... (read more)
Barkley Rosser, it's a strong assumption in principle, but in practice, humans seem to be pretty good at obtaining enough information to put in the model such that the posterior does in fact converge to some point in the parameter space.