Thus begins the ancient parable:

If a tree falls in a forest and no one hears it, does it make a sound? One says, “Yes it does, for it makes vibrations in the air.” Another says, “No it does not, for there is no auditory processing in any brain.”

If there’s a foundational skill in the martial art of rationality, a mental stance on which all other technique rests, it might be this one: the ability to spot, inside your own head, psychological signs that you have a mental map of something, and signs that you don’t.

Suppose that, after a tree falls, the two arguers walk into the forest together. Will one expect to see the tree fallen to the right, and the other expect to see the tree fallen to the left? Suppose that before the tree falls, the two leave a sound recorder next to the tree. Would one, playing back the recorder, expect to hear something different from the other? Suppose they attach an electroencephalograph to any brain in the world; would one expect to see a different trace than the other?

Though the two argue, one saying “No,” and the other saying “Yes,” they do not anticipate any different experiences. The two think they have different models of the world, but they have no difference with respect to what they expect will happen to them; their maps of the world do not diverge in any sensory detail.

It’s tempting to try to eliminate this mistake class by insisting that the only legitimate kind of belief is an anticipation of sensory experience. But the world does, in fact, contain much that is not sensed directly. We don’t see the atoms underlying the brick, but the atoms are in fact there. There is a floor beneath your feet, but you don’t experience the floor directly; you see the light reflected from the floor, or rather, you see what your retina and visual cortex have processed of that light. To infer the floor from seeing the floor is to step back into the unseen causes of experience. It may seem like a very short and direct step, but it is still a step.

You stand on top of a tall building, next to a grandfather clock with an hour, minute, and ticking second hand. In your hand is a bowling ball, and you drop it off the roof. On which tick of the clock will you hear the crash of the bowling ball hitting the ground?

To answer precisely, you must use beliefs like Earth’s gravity is 9.8 meters per second per second, and This building is around 120 meters tall. These beliefs are not wordless anticipations of a sensory experience; they are verbal-ish, propositional. It probably does not exaggerate much to describe these two beliefs as sentences made out of words. But these two beliefs have an inferential consequence that is a direct sensory anticipation—if the clock’s second hand is on the 12 numeral when you drop the ball, you anticipate seeing it on the 1 numeral when you hear the crash five seconds later. To anticipate sensory experiences as precisely as possible, we must process beliefs that are not anticipations of sensory experience.

It is a great strength of Homo sapiens that we can, better than any other species in the world, learn to model the unseen. It is also one of our great weak points. Humans often believe in things that are not only unseen but unreal.

The same brain that builds a network of inferred causes behind sensory experience can also build a network of causes that is not connected to sensory experience, or poorly connected. Alchemists believed that phlogiston caused fire—we could simplistically model their minds by drawing a little node labeled “Phlogiston,” and an arrow from this node to their sensory experience of a crackling campfire—but this belief yielded no advance predictions; the link from phlogiston to experience was always configured after the experience, rather than constraining the experience in advance.

Or suppose your English professor teaches you that the famous writer Wulky Wilkinsen is actually a “retropositional author,” which you can tell because his books exhibit “alienated resublimation.” And perhaps your professor knows all this because their professor told them; but all they're able to say about resublimation is that it's characteristic of retropositional thought, and of retropositionality that it's marked by alienated resublimation. What does this mean you should expect from Wulky Wilkinsen’s books?

Nothing. The belief, if you can call it that, doesn’t connect to sensory experience at all. But you had better remember the propositional assertions that “Wulky Wilkinsen” has the “retropositionality” attribute and also the “alienated resublimation” attribute, so you can regurgitate them on the upcoming quiz. The two beliefs are connected to each other, though still not connected to any anticipated experience.

We can build up whole networks of beliefs that are connected only to each other—call these “floating” beliefs. It is a uniquely human flaw among animal species, a perversion of Homo sapiens’s ability to build more general and flexible belief networks.

The rationalist virtue of empiricism consists of constantly asking which experiences our beliefs predict—or better yet, prohibit. Do you believe that phlogiston is the cause of fire? Then what do you expect to see happen, because of that? Do you believe that Wulky Wilkinsen is a retropositional author? Then what do you expect to see because of that? No, not “alienated resublimation”; what experience will happen to you? Do you believe that if a tree falls in the forest, and no one hears it, it still makes a sound? Then what experience must therefore befall you?

It is even better to ask: what experience must not happen to you? Do you believe that Élan vital explains the mysterious aliveness of living beings? Then what does this belief not allow to happen—what would definitely falsify this belief? A null answer means that your belief does not constrain experience; it permits anything to happen to you. It floats.

When you argue a seemingly factual question, always keep in mind which difference of anticipation you are arguing about. If you can’t find the difference of anticipation, you’re probably arguing about labels in your belief network—or even worse, floating beliefs, barnacles on your network. If you don’t know what experiences are implied by Wulky Wilkinsens writing being retropositional, you can go on arguing forever.

Above all, don’t ask what to believe—ask what to anticipate. Every question of belief should flow from a question of anticipation, and that question of anticipation should be the center of the inquiry. Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations. If a belief turns deadbeat, evict it.

Making Beliefs Pay Rent (in Anticipated Experiences)
New Comment
267 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

I assume that most of math is being ignored for simplicity's sake?

6David_Allencourt
I think his point isn't so much that what you're saying WILL have a practical impact on your sensory experiences, just that it has the potential to do so. What you "expect" to experience as a result. In real life we can't weld a pair of trillion-pound bars of gold to each other and then see how much they weigh, but because of mathematics, we know that if we were to place them on an accurate scale we would see a weight of two trillion pounds.

What good is math if people don't know what to connect it to?

All math pays rent.

For all mathematical theorems can be restated in the form:

If the axioms A, B, and C and the conditions X, Y and Z are satisfied, then the statement Q is also true.

Therefore, in any situations where the statements A,B,C and X,Y,Z are true, you will expect Q to also be verified.

In other words, mathematical statements automatically pay rent in terms of changing what you expect. (Which is) the very thing it was required to show. ■


In practice:

If you demonstrate Pythagoras's Theorem, and you calculate that 3^2+4^2=5^2, you will expect a certain method of getting right angles to work.

If you exhibit the aperiodic Penrose Tiling, you will expect Quasicrystals to exist.

If you demonstrate the impossibility of solving to the Halting Problem, you will not expect even a hypothetical hyperintelligence to be able to solve it.

If you understand why you can't trisect an angle with an unmarked ruler and a compass (not both used at the same time), you will know immediately that certain proofs are going to be wrong.

and so on and so forth.

Yes, we might not immediately know where a given mathematical fact will come in handy when observing the world, but by their nature, mathematical facts tell us exactly when to expect them.

5Daniel Clayton
Is this to say that one of the purposes of mathematics is to prove something new, even without knowing what it might be used for, with the awareness that it might be useful at a later point? Or that it might form part of a proof for something else that is also currently unknown? 
2MIN0010
Yes, there are numerous cases where a field in "pure" mathematics proved interesting theorems that mathematicians undertook because of its challenging and elegant nature (like certain theorems possess generality and elegance) which were then to be found to be practically useful, which are called "applied" mathematics. Frankly, this distinction is blurred as pure mathematics are so useful (see Eugene Wigner's "The Unreasonable Effectiveness of Mathematics in the Natural Sciences") that the abstract nature of mathematics has huge extensibility and general applications in multiple domains. For instance, Einstein's GR was based on the pure mathematics of Riemannian manifolds, which is an abstract topological structure, not tied to reality in any way initially. Or how algebraic topology is used for data mining, how number theory is used for cryptography, how linear algebra is used for machine learning, group theory is used for particle physics... and even how Bayesian probability theory is used for LW rationality. Stephen Wolfram has great resources on rulial spaces and the nature of computation for the universe's fundamental ontology (the territory not the map) in which these networks of theorems can correspond to our empirical reality. (psst I am a very new LW user, and I am deciding if I should do a Sequence for this idea of "rulial cover" which is how rulial deduction can be applied to Solomonoff induction and Bayesian abduction, would be great if someone thinks this is interesting to explore so I can be motivated) To link back to Eliezer's post, "floating beliefs" in a Bayesian net can be connected through adjusting the "weights" of the edges that connects that belief using Bayesian inference, and mathematics make these robust inferences from axioms (deductively validity as 100% in weight and 0% in prior). Therefore, anticipation becomes certain under a set of idealized axioms.
2Bruno Vieira
'I am deciding if I should do a Sequence for this idea of "rulial cover" which is how rulial deduction can be applied to Solomonoff induction and Bayesian abduction'   I don't really know what you mean, but if it's something unseen you can expect it to be useful!
6jirkazr
Is it not the purpose of math to tell us "how" to connect things? At the bottom, there are some axioms that we accept as basis of the model, and using another formal model we can infer what to expect from anything whose behavior matches our axioms. Math makes it very hard to reason about models incorrectly. That's why it's good. Even parts of math that seem particularly outlandish and disconnected just build a higher-level framework on top of more basic concepts that have been successfully utilized over and over again. That gives us a solid framework on which we can base our reasoning about abstract ideas. Just a few decades ago most people believed the theory of probability was just a useless mathematical game, disconnected from any empirical reality. Now people like you and me use it every day to quantify uncertainty and make better decisions. The connections are not always obvious.
4A1987dM
http://abstrusegoose.com/504 :-)
0[anonymous]
Thats exactly how i felt in high school. Im glad i changed that because it wouldn't be useful to me if i'd never learned algebra. The first part of the class is hard to use and discouraging to new students.
-4TheAncientGeek
Is pure math a set of beliefs that should be evicted?
4g_pepper
No, for reasons expressed above by VKS.
0TheAncientGeek
Note the word "pure". By definition, pure maths doesn't pay off in experience. If it did, it would be applied.
2g_pepper
IMO the distinction between pure and applied math is artificial, or at least contingent; today's pure math may be tomorrow's applied math. This point was made in VKS's comment referenced above:
0TheAncientGeek
The question is whether anyone should believe pure maths now. If you are allowed to believe things that might possibly pay off, then the criterion excludes nothing.
3lalaithion
Metabeleifs! Applied math concepts that seem useless now, have, in the past, become useful. Therefore, the belief that "believing in applied math concepts pays rent in experience" pays rent in experience, so therefore you should believe it.
0g_pepper
Unlike scientific knowledge or other beliefs about the material world, a mathematical fact (e.g. that z follows from X1, X2,..., Xn), once proven, is beyond dispute; there is no chance that such a fact will be contradicted by future observations. One is allowed to believe mathematical facts (once proven) because they are indisputably true; that these facts pay rent is supported by VKS's argument.
1TheAncientGeek
Truths of pure maths don't pay rent in terms iof expected experience. EY has put forward a criterion of truth, correspondence, and a criterion of believability, expected experience , and pure maths fits neither. He didn't want that to happen, and the problem remains, here and elsewhere, of how to include abstract maths and still exclude the things you don't like. This is old ground, that the logical postivists went over in the mid 20th century.
0Richard_Kennaway
Here is a truth of pure mathematics: every positive integer can be expressed as a sum of four squares. Expected experiences: there will be proofs of this theorem, proofs that I can follow through myself to check their correctness. Et voilà!
1TheAncientGeek
Truth of astrology: mars in conjunction with Jupiter is dangerous for Leos Expected experience: there will be astrology articles saying Leo's are in danger when mars is in conjunction with Jupiter.
3Richard_Kennaway
Of course astrological claims pay rent. The problem with astrology is not that it's meaningless but that it's false, and the problem with astrologers is that they don't pay the epistemological rent. Also, a proof is a different thing from a mathematician saying so. The rent that is being paid there is not merely that the theorem will be asserted but that there will be a proof.
0TheAncientGeek
Try telling Eliezer
0Richard_Kennaway
The original post does not mention astrology. If you want to spy out some place where Eliezer has said that astrological claims are meaningless, go right ahead. I am not particularly concerned with whether he has or not. Here and now, you are talking to me, and as I pointed out, the belief can pay rent, but astrologers are not making it do so. Those who have seriously looked for evidence, have, so I understand, generally found the beliefs false.
0polymathwannabe
From that belief, the expected experience should be Leo people being less fortunate during those days.
0TheAncientGeek
That was the point. Its a cheat to expect astrology truths to product experiences of reading written materials about astrology, so it's a cheat expect to pure maths truths ...
3Richard_Kennaway
Let me complete the ellipsis with what I actually said. A mathematical assertion leads me to expect a proof. Not merely experiences of reading written materials repeating the assertion.
0TheAncientGeek
And a proof still isnt an .experience in the relevant sense. Its not like predicting an eclipse,
1wizzwizz4
What's the difference between behaviours of non-sentient objects and behaviours of sentient people that makes one an experience and the other not?
1Paolo Falabella
I think this is both right and not in contradiction with the post. The belief that pays the rent here is that there is going to be a high correlation between Mars being in conjunction with Jupiter and astrology believers born around August experiencing heightened feelings of being in danger. That does not say anything on the "truth" of astrology itself. Same applies to the article's example on Wulky Wilkinsen. The belief that alienated resublimation justifies the fictional author's retropositionality does not pay rent. The belief that failing to mention retropositionality correlates with higher chances of failing a literature test on Wilkinsen does probably pay rent.
1g_pepper
I think I see where you are going with this. My initial interpretation of EY's original post is that he was explicating a scientific standard of belief that would make sense in many situations, including in reasoning about the physical world (EY's initial examples were physical phenomena - trees falling, bowling balls dropping, phlogiston, etc.). I did not really think he was proposing the only standard of belief. This is why I was baffled by your insistence that unless a mathematical fact had made successful predictions about physical, observable phenomena, it should be evicted. However, later in the original post EY used an example out of literary criticism, and here he appears to be applying the standard to mathematics. So, you may be on to something - perhaps EY did intend the standard to be universally applied. It seems to me that applying EY's standard too broadly is tantamount to scientism (which I suspect is more-less the point you were making).
0Epictetus
If you believe in applied math, what are the grounds for excluding "pure" math? Most of the time "pure" just means that the mathematician makes no explicit reference to real-world applications and that the theorems are formulated in an abstract setting. Abstraction usually just boils down to figuring out exactly which hypotheses are necessary to get the conclusion you want and then dispensing with the rest. Let's take the theory of probability as an example. There's nothing in the general theory that contradicts everyday, real-world probability applications. Most of the time the general theory does little other than make precise our intuitive notions and avoid the paradoxes that plague a naive approach. This is an artifact of our insistence on logic. A thorough, logical examination of just about any piece of mathematics will quickly lead to the domain "pure" math.
-1TheAncientGeek
I am not making the statement "exclude pure math", I am posing the question "if pure math stays, what else stays?" Maybe post utopianism is an abstract idealisation that makes certain concepts precise.
3Epictetus
There are beliefs that directly pay rent, and then there are beliefs that are logical consequences of rent-paying beliefs. The same basic principles that give you applied math will also lead to pure math. We can justify spending effort on pure math on the grounds that it may pay off in the future. However, our belief in pure math is tied to our belief in logic. If you asked whether this can be applied to something like astrology, I'd ask whether astrology was a logical consequence of beliefs that do pay rent.

In practice, most of the time people figure out what to connect it to later. More precisely, most of it probably doesn't connect to anything, but what does connect to stuff usually isn't found to do so until much later than it is invented/discovered.

Some ungrounded concepts can produce your own behavior which in itself can be experienced, so it's difficult to draw the line just by requiring concepts to be grounded. You believe that you believe in something, because you experience yourself acting in a way consistent with you believing in it. It can define intrinsic goal system, point in mind design space as you call it. So one can't abolish all such concepts, only resist acquiring them.

For any instrumental activity, done to achieve some other end, it makes sense to check that specific examples are in fact achieving the intended end.

Most beliefs may have as their end the refinement of personal decisions. For such beliefs it makes sense not only to check whether they effect your personal experience, but also whether they effect any decisions you might make; beliefs could effect experience without mattering for decisions.

On the other hand, some beliefs may have as their end effecting the experiences or decisions of other creatures, such as in the far future. And you may care about effects that are not experienced by any creatures.

0[anonymous]
Only if you have reason to believe your naive pattern matching of expectations to observation isn't already updating your expectations about instrumental activity. Otherwise, your ''privileging the hypothesis'' that you are in fact wrong. It's kind of like smoothing in machine learning. It will have costs and benefits.

Elizer, your post above strikes me, at least, as a restatement of verificationism: roughly, the view that the truth of a claim is the set of observations that it predicts. While this view enjoyed considerable popularity in the first part of the last century (and has notable antecedents going back into the early 18th century), it faces considerable conceptual hurdles, all of which have been extensively discussed in philosophical circles. One of the most prominent (and noteworthy in light of some of your other views) is the conflict between verificationism and scientific realism: that is, the presumption that science is more than mere data-predictive modeling, but the discovery of how the world really is. See also here and here.

1[anonymous]
Maybe I'm inferring from too little data, but I suspect that most readers at this site aren't too interested in sceintific realism. Our favourite mantra ("the map is not the territory") acknowledges and then gracefully side-steps the issues that you're raising. (I just realized that Eliezeer answers this below. Comment retracted. Is there some way for me to delete this?)

Rooney, as discussed in The Simple Truth I follow a correspondence theory of truth. I am also a Bayesian and a believer in Occam's Razor. If a belief has no empirical consequences then it could receive no Bayesian confirmation and could not rise to my subjective attention. In principle there are many true beliefs for which I have no evidence, but in practice I can never know what these true beliefs are, or even focus on them enough to think them explicitly, because they are so vastly outnumbered by false beliefs for which I can find no evidence.

9Perplexed
I, too, am nervous about having anticipated experience as the only criterion for truth and meaning. It seems to me that a statement can get its meaning either from the class of prior actions which make it true or from the class of future observations which its truth makes inevitable. We can't do quantum mechanics with kets, but no bras. We can't do Gentzen natural deduction with rules of elimination, but no rules of introduction. We can't do Bayesian updating with observations, but no priors. And I claim that you can't have a theory of meaning which deals only with consequences of statements being true but not with what actions put the universe into a state in which the statement becomes true. This position of mine comes from my interpretation of the dissertation of Noam Zeilberger of CMU (2005, I think). Zeilberger's main concern lies in Logic and Computer Science, but along the way he discusses theories of truth implicit in the work of Martin-Lof and Dummett.
0timtyler
That seems obviously correct. However, unless you pursue knowledge for its own sake, you should probably not be overly concerned with preserving past truths - unless they are going to impact on future decisions. Of course, the decisions of a future superintelligence might depend on all kinds of historical minutae that we don't regard as important. So maybe we should preserve those truths we regard as insignificant to us for it. However, today, probably relatively few are enslaved to future superintelligences - and even then, it isn't clear that this is what they would want us to do.
1Peter Pehlivanov
Perplexed, I'm not sure I understood what you meant by Or if I agree with it at all. Wouldn't statements about what actions make certain statements true simply be part of the first category? I don't see a problem with only having statements and their consequences. I see you've made this comment 12 years ago, so I don't know how you would stand on this today.
2mendel
An explicit belief that you would not allow yourself to hold under these conditions would be that the tree which falls in the forest makes a sound - because no one heard it, and because we can't sense it afterwards, whether it made sound or not had no empirical consequence. Every time I have seen this philosophical question posed on lesswrong, the two sophists that were arguing about it were in agreement that a sound would be produced (under the physical definition of the word), so I'd be really surprised if you could let go of that belief.
1Manfred
Hm, yeah. The trouble is how the doctrine handles deductive logic - for example, the belief that a falling tree makes vibrations in the air when the laws of physics say so is really a direct consequence of part of physics. The correct answer definitely appears to be that you can apply logic, and so the doctrine should be not to believe in something when there is no Bayesian evidence that differentiates it from some alternative.
0Ty-Guy9
While I fully agree with the principle of the article, something stuck out to me about your comment: What I noticed was that you were basically defining a universal prior for beliefs, as much more likely false than true. From what I've read about Bayesian analysis, a universal prior is nearly undefinable, so after thinking about it a while, I came up with this basic counterargument: You say that true beliefs are vastly outnumbered by false beliefs, but I say, how could you know of the existence of all these false beliefs, unless each one had a converse, a true belief opposing it that you first had some evidence for? For otherwise, you wouldn't know whether it was true or false. You may then say that most true beliefs don't just have a converse. They also have many related false beliefs opposing them. But I would say, those are merely the converses that spring from the connections of that true belief with its many related true beliefs. By this, I hope I've offered evidence that a fifty-fifty universal T/F prior is at least as likely as one considering most unconsidered ideas to be false. (And I would describe my further thoughts if I thought they would be useful here, but, silly me, I'm replying to a post from almost 8 years ago.)
1CBHacking
I don't think "converse" is the word you're looking for here - possibly "complement" or "negation" in the sense that (A || ~A) is true for all A - but I get what you're saying. Converse might even be the right word for that; vocabulary is not my forte. If you take the statement "most beliefs are false" as given, then "the negation of most beliefs is true" is trivially true but adds no new information. You're treating positive and negative beliefs as though they're the same, and that's absolutely not true. In the words of this post, a positive belief provides enough information to anticipate an experience. A negative belief does not (assuming there are more than two possible beliefs). If you define "anything except that one specific experience" as "an experience", then you can define a negative belief as a belief, but at that point I think you're actually falling into exactly the trap expressed here. If you replace "belief" with "statement that is mutually incompatible with all other possible statements that provide the same amount of information about its category" (which is a possibly-too-narrow alternative; unpacking words is hard sometimes) then "true statements that are mutually incompatible with all other possible statements that provide the same amount of information about their category are vastly outnumbered by false statements that are mutually incompatible with all other possible statements that provide the same amount of information about their category" is something the I anticipate you would find true. You and Eliezer do not anticipate a different percentage of possible "statements that are mutually incompatible with all other possible statements that provide the same amount of information about their category" being true. As for universal priors, the existence of many incompatible possible (positive) beliefs in one space (such that only one can be true) gives a strong prior that any given such belief is false. If I have only two possible beliefs and
2gjm
If you have an arbitrary proposition -- a random sequence of symbols constrained only by the grammar of whatever language you're using -- then perhaps it's about equally likely to be true or false, since for each proposition p there's a corresponding proposition not p of similar complexity. But the "beliefs" people are mostly interested in are things like these: * There is exactly one god, who created the universe and watches over us; he likes forgiveness, incense-burning, and choral music, and hates murder, atheism and same-sex marriage. * Two nearby large objects, whatever they are, will exert an attractive force on one another proportional to the mass of each and inversely proportional to the square of the distance between them. and the negations of these are much less interesting because they say so much less: * Either there is no god or there are multiple gods, or else there is one god but it either didn't create the universe or doesn't watch over us -- or else there is one god, who created the universe and watches over us, but its preferences are not exactly the ones stated above. * If you have two nearby objects, whatever force there may be between them is not perfectly accurately described by saying it's proportional to their masses, inversely proportional to the square of the distance, and unaffected by exactly what they're made of. So: yeah, sure, there are ways to pick a "random" belief and be pretty sure it's correct (just say "it isn't the case that" followed by something very specific) but if what you're picking are things like scientific theories or religious doctrines or political parties then I think it's reasonable to say that the great majority of possible beliefs are wrong, because the only beliefs we're actually interested in are the quite specific ones.

It's amazing how many forms of irrationality failure to see the map-territory distinction, and the resulting reification of categories (like 'sound') that exist in the mind, causes: stupid arguments, phlogiston, the Mind Projection Fallacy, correspondence bias, and probably also monotheism, substance dualism, the illusion of the self, the use of the correspondence theory of truth in moral questions... how many more?

I think you're being too hard on the English professor, though. I suspect literary labels do have something to do with the contents of a book, no matter how much nonsense might be attached to them. But I've never experienced a college English class; perhaps my innocent fantasies will be shaken then.

Michael V, you could say that mathematical propositions are really predictions about the behavior of physical systems like adding machines and mathematicians. I don't find that view very satisfying, because math seems to so fundamentally underly everything else - mathematical truths can't be changed by changing anything physical, for instance - but it's one way to make math compatible with anticipation.

9TsviBT
I think Eliezer's point was about the student. "Wulky Wilkinsen is a 'post-utopian'" could be meaningful, if you know what a post-utopian is and is not (I don't, and don't care). The student who learns just the statement, however, has formed a floating belief. We might even initially use propositional beliefs as indicators of meaningful beliefs about the world. But if we then discuss these highly compressed beliefs without referencing their meaning, we often feel like we are reasoning when really we have ceased to speak about the world. That is, grounded beliefs can become "floaty" and spawn further "floaty" beliefs. In my sociology class, we talk about how "Man in his natural state has liberty because everyone is equal". "Natural state", "liberty", and "equal" could conceivably be linked to descriptions of social interaction or something. However, class after class we refrain from talking about specific behaviors. Concepts float away from their referents without much resistance - it's all the same to the student, who only needs to make a few unremarkable remarks to get his B+ for class participation. Compare: "Man in his natural state has liberty because everyone is equal" "Man in his natural state is equal because everyone has liberty" "When everyone has liberty and is equal, man is in his natural state" These statements should express very different beliefs about the world, but to the student they sound equally clever coming out of the professor's mouth. (Edit for minor grammar and formatting)

It's amazing how many forms of irrationality failure to see the map-territory distinction

Should have been "how many forms of irrationality result from failure...". Sorry.

I agree with those who say it's okay to figure things out later. If my music professor says a certain composer favors the Aeolian mode, I may not be able to visualize that on the spot but who cares? I can remember that statement and think about it later. Likewise with phlogiston, I have a vague concept of what it is and someday the alchemists will discover more precisely what's going on there.

Too much cognitive effort would be spent if, every time I thought about linear algebra, I had to visualize the myriad concrete instances in which it will be applied. I bet thinking in abstractions results in way more economical use of thinking time and thinking-matter.

In what way is the belief that beliefs should be grounded not a free-floating belief itself?

1adamisom
One way of answering might be to say that there is no separate "belief" that beliefs should be grounded. But i'm not sure. All I know is that the question annoys me, but I can't quite put my finger on it. It reminds me of questions like (1) the accusation that you can't justify the use of logic logically, or (2) the accusation that tolerance is actually intolerant - because it's intolerant of intolerance. There might be a level distinction that needs to be made here, as in (2) - and maybe in (1) though I think that's different.
1Danfly
(1) has come out of my mouth on a few occasions, albeit not in those exact words. It's normally after a few beers and I feel like playing the extreme skeptic a la David Hume, just to annoy everyone. I think the best way around it is to resort to the empirical argument and say that, in our experience, it is always right: Essentially the same thing Yudkowsky does with PA arithmetic here. Trying to find an argument against it which is truly "rationalist" in the continental sense has been a dead end in my experience. (2) sort of depends on the pragmatics and what "tolerance" actually means to the persons involved in a given context. If you define tolerance as simply being tolerant of other viewpoints, then you can still be tolerant of the intolerant viewpoints. However, if you define it as freedom from bigotry, then that could indeed be called "intolerant" by the standards of the first definition. I hope I'm making sense here.
1MarkusRamikin
I anticipate expressing free-floating beliefs would get me negative karma on Less Wrong. More seriously: I do not anticipate free-floating beliefs being useful in the same sense that maps of reality are useful. A map can turn out to be accurate or inaccurate, and insofar as it is accurate it can help me navigate and manipulate reality. My belief that "a proper belief should not be free-floating" prohibits free-floating beliefs from doing any of that. Or one might as well see it as not a belief, but as a definition. There's BeliefType1 which is grounded in reality, and BeliefType2 which is not, and we happen to call BeliefType1 a "proper belief". (Of course we still do it for a reason, because we care about our sheep, or rather, we care about our beliefs being true and thus useful.) Not sure which approach makes more sense.
0Klevador
The ability to anticipate experiences is one of our maximands because we have goals that are optimally achieved with this ability. To believe that beliefs should allow us to anticipate experiences is grounded in the desire to achieve our goals.

Mark: Believing that beliefs should be grounded anticipates that there is absolutely no change in anticipation if one were to change these free floating ideas. Of course this doesn't really answer your question because it just restates the definition of 'free floating beliefs' in different words. This belief actually follows from Eliezers belief in Occam's Razor, which predicts that when faced with unexplained events, if one creates a set of theories explaining these events, any predictions made by the simple theories are more likely to actually happen tha... (read more)

Jan: Occam's razor is not so much a rule of science but an operating guideline for doing science. It could be reduced to "test simple theories first". In the past this has been very useful in keeping scientific effort productive, the 'belief' is that it will continue to be useful in this way.

This led to a fun read of "occam's razor" wikipedia entry. Hickum's dictum in particular was a great find (generalized beyond medicine, it could be that explanations for unexplained events can be as complex as they damn well please). As a practical corrective, it seems to me that probability theory suggests that the best accessible explanation to us for unexplained events is in the set of simpler theories, but is probably not one of the absolute simplest.

Eliezer once wrote that "We can build up whole networks of beliefs that are connected only to each other - call these "floating" beliefs. It is a uniquely human flaw among animal species, a perversion of Homo sapiens's ability to build more general and flexible belief networks.

The rationalist virtue of empiricism consists of constantly asking which experiences our beliefs predict - or better yet, prohibit."

I can't see how nearly all of the beliefs expressed in this post predict or prohibit any experience.

"Alchemists believed that phlogiston caused fire"

How is that different than our current belief that oxygen causes fire?

-1Jack
Uhhh... oxygen exists?
1DanielLC
And so does the absence of oxygen, or, as they called it, phlogiston.
9Nick_Tarleton
The absence of oxygen isn't much like a substance whose release is fire: * it doesn't have any consistent physical or chemical properties; * many things not containing oxygen fail to burn in air, and none burn in vacuum; * on the other hand, things do burn under oxidizers other than oxygen; * oxidized substances are very poorly modeled by mixtures of the original substance and oxygen; * things burned in open air can either gain or lose weight; etc.
1DanielLC
"it doesn't have any consistent physical or chemical properties;" And oxides do? Or are you referring to pure phlogiston? It's not that big a deal that you can't get pure phlogiston. It's nigh impossible to purify fluorine. I think that under our current understanding of physics, it's totally impossible to isolate a single quark. It moves because it's attracted to some things more than others. It's still attracted to everything more than itself. "many things not containing oxygen fail to burn in air" Hurts both theories equally. Presumably, it's strongly bonded to the phlogiston/it doesn't strongly bond to oxygen. "...and none burn in vacuum;" As I said, you can't get pure phlogiston. "on the other hand, things do burn under oxidizers other than oxygen;" Hurts both theories equally. The only way to solve it to my knowledge is that there are things that cause fire other than phlogiston/oxygen. "things burned in open air can either gain or lose weight;" Hurts both theories equally. Presumably, some of the matter escapes into the air sometimes. Everything you listed either is only a very minor problem or is exactly as bad for the idea of oxygen.

You're giving phlogiston qualities no one who held that theory gave it. If you want to call the absence of oxygen phlogiston, okay, but you aren't talking about the same phlogiston everyone else is talking about. Moreover, thinking about fire this way is clumsy and incompatible with the rest of our knowledge about physics and chemistry.

We already had a conception of matter when phlogiston was invented... and phlogiston was understood as a kind of matter. To say the phlogiston is really this other kind of thing, which isn't matter but a particular kind of absence of matter is both unhelpful and a distortion of phlogiston theory. The whole point of the phlogiston theory was that they thought there was a kind of matter responsible for fire! But there isn't matter like that.

Now by defining phlogiston as the absence of oxygen you might be able to model combustion in a narrow set of circumstances-- but you couldn't fit that model with any of your other knowledge about physics and chemistry.

In short neither the original kind nor your kind of phlogiston exist.

1DanielLC
It was at one point theorized to have negative mass. If it's matter, and you make everything else weigh more, it works out the same. I fail to see why you think it can't fit it with other knowledge of physics and chemistry. You can think of electricity as positively charged particles moving around with virtually zero loss of predicting power.
4Jack
For example, you can't use phlogiston in any model that also includes oxygen. Nor can you do any work at the molecular or sub-molecular level. Similarly, thinking of electricity in terms of positively charged particles would be incompatible with atomic theory.
-6DanielLC
4Sniffnoy
Because one of these allows you to make predictions, and the other doesn't. Saying "fire has a cause, and I'm going to call it 'phlogiston'!" doesn't tell you anything about fire, it's just a relabeling. Now, if you make enough observations, maybe you'll eventually conclude that "phlogiston is the absence of oxygen" (even though this isn't really correct), but at that point you can throw out the label "phlogiston". Contrariwise, if you say "oxidization causes fire", where "oxygen" is a previously known thing with known properties, then this allows you to actually make predictions about fire. E.g. the fact a candle in a sufficiently small closed space will go out before it melts, but not necessarily if there's a plant in there too. One pays rent, the other doesn't.
3DanielLC
You can make exactly the same predictions with phlogiston. If you burn coal next to iron, it will refine it. You could predict this with oxygen (oxygen is moving from the iron to the coal) or with phlogiston (phlogiston is moving from the coal to the iron). It's like with electric charge. If you think of it as positive charge moving around, it has almost exactly the same predictive power as thinking of it as electrons moving around.
0[anonymous]
In this specific example and at that level of precision, yes; but only one of these models can be (easily) refined to make precise, correct quantitative predictions. Even at that qualitative level, though, they make different predictions about burning things in vacuum or in non-oxygen atmospheres.
3Sniffnoy
But you can only predict it if you already know that a gain of phlogiston refines iron; if you don't, you can only observe it afterward and write it down as a property of phlogiston. If you don't know anything about oxygen or phlogiston beforehand, then, sure, they're pretty much equally predictive, i.e., not very much. But if "oxygen" is not in fact just an arbitrary label as "phlogiston" is, but in fact something you're already working with in other ways, then they're not symmetric. Also as Nick Tarleton points out below there are other asymmetries, though those are not so much in the predictive power.
0DanielLC
"But you can only predict it if you already know that a gain of phlogiston refines iron" Same goes for oxygen.
-1DanielLC
Okay, I admit that that's not really a prediction, but until then, they couldn't even explain it. If you're going to do it like this, what's one thing oxygen predicted? By the way, I'm responding to the fact that I lost two karma points on that, not any actual post.
3Sniffnoy
That's what I just said.
3DanielLC
Sorry. Too used to defending my position to realize you're not attacking it.

Because one of these allows you to make predictions, and the other doesn't. Saying "fire has a cause, and I'm going to call it 'phlogiston'!" doesn't tell you anything about fire, it's just a relabeling.

The hypothesis went a little deeper than that. "Flammable things contain a substance, and its release is fire" lets you make many predictions — e.g., that things will burn in vacuum, or that things burned in open air will always lose mass (this is how it was falsified).

0Sniffnoy
Ah, true.
-1DanielLC
Always gain mass, once they realized it was negative mass. The idea that it doesn't always gain mass doesn't falsify phlogiston any more than it falsifies oxygen for the same reason. Also, people didn't find the change in weight particularly useful, so this wasn't that big a problem. Again, the vacuum thing isn't much of a problem either. It's not necessarily possible to purify phlogiston.
4bigjeff5
I'm not sure I follow, oxidization doesn't predict gaining or losing mass (on any scale like phlogiston would, that is), it predicts an interaction of materials forming a new composite substance. Oxidation doesn't prevent material from being lost or changed in other ways which could cause an overall greater or lesser mass than the original object. What it does predict, however, is that the total mass of all molecules in the equation, once accounted for, will be the same. This is consistent with observation. If phlogiston has a negative mass, then anything that can burn must gain mass. I don't see any way around it. The theory states that it is a release of negative material, and there is no way to account for it once released. One thing you would expect to find with phlogiston is an object that was primarily made up of phlogiston, giving it a negative mass. Explosives, for example, clearly have so much phlogiston that it literally rips the object (and anything nearby) apart when released. You would therefore expect all explosives to be relatively light in spite of the original weight of their components. You could test this with black powder: saltpeter, charcoal, and sulfer each release a certain amount of phlogiston when burned. Combine them and a significantly more phlogiston is clearly released. You would therefore expect more phlogiston to have flowed into the material during the combination of the three objects during the making of gunpowder. However, the weights actually stay quite the same. The observation doesn't bear out the prediction, so the prediction is clearly wrong. If the prediction is wrong, the theory that made it is either wrong outright, or flawed in some way. Since the only prediction phlogiston can make is wrong, then the theory is at the very least flawed in some crippling way, and needs to be completely re-worked. It's lack of ability to predict expectations is what killed it. You can predict what will happen when you add oxygen to a reac
2thomblake
Just because I haven't seen the link in this particular discussion, some more defense of phlogiston link

I loved this post, but I have to be a worthless pedant.

If you drop a ball off a 120-m tall building, you expect impact in t=sqrt(2H/g)=~5 s. But that would be when the second-hand is on the 1 numeral.

8Eliezer Yudkowsky
Heh. I got this right originally, then reread it just recently while working on the book, saw what I thought was an error (1 numeral? just one second? why?) and "fixed" it.

What about knowledge for the sake of knowledge? For instance I don't anticipate that my belief that The Crusades took place will ever directly affect my sensory experiences in any way. Does that then mean that this belief is completely worthless and on the same level as the belief in ghosts, psychics, phlogiston, etc.?

Wouldn't taking your chain of reasoning to its logical conclusion require one to "evict" all beliefs in everything that one has not, and does not anticipate to, personally see, hear, smell, taste, or touch? After all, how much personal sensory experience do you have that confirms the existence of atoms, for example?

DP

6RobinZ
I think Eliezer's point is less strong than you think: for one thing, reading a history book is a sensory experience, and fewer history books would proclaim that The Crusades occurred in worlds where they had not than in worlds where they had.
0Dpar
I was going to write a more detailed reply, but then realized that any continued discussion will require us to debate what exactly the OP meant to say in his post, which is pointless since neither of us can read his mind. So let's just call it a day. DP
3Vladimir_Nesov
This is something of a fallacy of gray. Of course we can read his mind, through the power of human telepathy, by reading more on the same topic. We can't read minds perfectly, but perfect knowledge is never available anyway, and unless you can point out the specific uncertainty you have that decides the discussion, there is no sense in requiring more detail. You might want to stop the discussion for other reasons, but the reason you stated rings false.
2Dpar
First of all, calling speech "human telepathy" strikes me as a little pretentious, as well as inaccurate, since the word "telepathy" is generally accepted to have supernatural connotations. Speech is speech; no need to complicate the concept. Secondly, the article you linked seemed a little rambling and without a clear point. All I was able to take away from it is that the meaning of words is relative. If that's the case then I respond with "well, duh!"; if I missed a deeper point, please enlighten me. Finally, when you take it upon yourself to question another person's purely subjective reasoning, you're treading very close to completely indefensible territory. If I say that I wanted to stop the discussion because I believe that the author's intended meaning is ambiguous, it's a tall order to question that that is indeed what I believe. Unless you can come up with clear evidence of how my behavior contradicts my stated subjective opinion, you more or less have to take my word that that really is what I think. DP
3thomblake
You misunderstand. Vladimir Nesov was not claiming that you don't believe that the author's intended meaning is ambiguous. Rather, he was claiming that your belief that "the author's intended meaning is ambiguous" is false, or at least not enough to constitute a good reason for stopping the discussion. The point of calling speech 'human telepathy' in this instance is that you claimed there's no way to know what the author was thinking since we "can't read his mind". But there is a way to know what the author was thinking to some extent, so by reading your own reasoning backwards we therefore indeed can read minds.
-1Dpar
I stated that taking the OP's reasoning to its logical conclusion requires one to "evict" all beliefs in everything that one has not, and does not anticipate to, personally see, hear, smell, taste, or touch. RobinZ responded by saying that the OP's point is less strong than I think. Since two (presumably) reasonable people can disagree on what the OP meant, his point, as it is written, is by definition ambiguous. Where do we go from here other than debate what he really meant? What is the point of such debate since neither of us has any special insight into his thought process that would allow us to settle this difference of subjective interpretations? I believe that to be sufficient reason for stopping the discussion. I'm not sure what specifically Vladimir takes issue with here. As to your point of human telepathy -- comparing reading what someone wrote to reading his mind is a very big stretch. I can see how you could make that argument if you get really technical with word definitions, but I think that it is generally accepted that reading what a person wrote on a computer screen and reading his mind are two very different things. DP
5thomblake
Right, but RobinZ was not arguing against this claim (depending on what you mean by 'personally' here) but rather pointing out that your reasoning was flawed. RobinZ pointed out that your belief that the crusades took place affects your sensory experience; if you believe they happened, then you should anticipate having the sensory experience of seeing them in the appropriate place in a history book, if you were to check. If you thought that your belief that the crusades happened did not imply any such anticipated experiences, then yes, it would be worthless and on the same level as belief in an invisible dragon in your garage.
0Dpar
So reading about something in a book is a sensory experience now? I beg to differ. A sensory experience of The Crusades would be witnessing them first hand. The sensory experience of reading about them is perceiving patterns of ink on a piece of paper. DP Edit: Also, I think that RobinZ didn't state that as something that she believed, she stated that as something that she believed the OP meant. It's that subjective interpretation of his position that I didn't want to debate. If you wish to adapt that position as your own and debate its substance, we certainly can.
4Oligopsony
What's important isn't the number of degrees of removal, but that the belief's being true corresponds to different expected sensory experiences of any kind at all than its being false. The sensory experience of perceiving patterns of ink on a piece of paper counts. Now you could say: "reading about the Crusades in history books is strong evidence that 'the Crusades happened' is the current academic consensus," and you could hypothesize that the academic consensus was wrong. This further hypothesis would lead to further expected sensory data - for instance, examining the documents cited by historians and finding that they must have been forgeries, or whatever.
-5Dpar
0Vladimir_Nesov
You are disputing definitions. Reading something in a book is a sort of thing you'd change expectation about depending on your model of the world, as are any other observations. If your beliefs influence your expectation about observations, they are part of your model of reality. On the other hand, if they don't, they are sometimes too part of your model of reality, but it's a more subtle point. And returning to your earlier concerns, consider me having a special insight into the intended meaning, and proving counterexample to the impossibility of continuing the discussion. Reading something in a history book definitely counts as anticipated experience.
1Dpar
Very interesting read on disputing definitions. While the solution proposed there is very clever and elegant, this particular discussion is complicated by the fact that we're discussing the statements of a person who is not currently participating. Coming up with alternate words to describe our ideas of what "sensory experience" means does nothing to help us understand what he meant by it. Incidentally this is why I didn't want to get drawn into this debate to begin with. Also -- "consider me having a special insight into the intended meaning" -- on what grounds shall I consider your having such special insight?
2Cyan
At the bottom of the sidebar at the bottom, you will find a list of top contributors; Vladimir Nesov is on the list.
2Vladimir_Nesov
I've closely followed Yudkowsky's work for a while, and have a pretty good model of what he believes on topics he publicly discusses.
0Dpar
Fair enough. So if, on your authority, the OP believes that reading about something is anticipated experience, does that not then cover every rumor, fairy tale, and flat out non-sense that has ever been written? What then would be an example of a belief that CANNOT be connected to an "anticipated experience"?
3Vladimir_Nesov
See this comment on the first part of your question and this page on the second (but, again, there are valid beliefs that don't translate into anticipated experience).
1Dpar
I agree wholeheartedly that there are valid beliefs that don't translate into anticipated experience. As a matter of fact what's written there was pretty much the exact point that I was trying to make with my very first response in this topic. Does that not, however, contradict the OP's assertion that "Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations. If a belief turns deadbeat, evict it."? That's what I took issue with to begin with.
2Vladimir_Nesov
It does contradict that assertion, but not at first approximation, and not in the sense you took the issue with. You have to be very careful if a belief doesn't translate into anticipated experience. Beliefs about historical facts that don't translate into anticipated experience (or don't follow from past experience, that is observations) are usually invalid.
0Dpar
You seem to place a good deal of value on the concept of anticipated experience, but you give it a definition that's so broad that the overwhelming majority of beliefs will meet the criteria. If the belief in ghosts for instance can lead to the anticipated experience of reading about them in a book, what validity does the notion have as a means of evaluating beliefs?
2Vladimir_Nesov
When a belief (hypothesis) is about reality, it responds to new evidence, or arguments about previously known evidence. It's reasonable to expect that as a result, some beliefs will turn out incorrect, and some certainly correct. Either way it's not a problem: you do learn things about the world as a result, whatever the conclusion. You learn that there are no ghosts, but there are rainbows. The problem are the beliefs that purport to be speaking about reality, but really don't, and so you become deceived by them. Not being connected to reality through anticipated experience, they take your attention where there is no use for them, influence your decisions for no good reason, and protect themselves by ignoring any knowledge about the world you obtain. It is a great heuristic to treat any beliefs that don't translate into anticipated experience with utmost suspicion, or even to run away from them in horror.
0Dpar
How would you learn that there are no ghosts? You form the belief "there are ghosts" which leads to the anticipated experience (by your definition of such) that "I will read about ghosts in a book", you go and read about ghosts in a book. Criteria met, belief validated. Same goes for UFOs, psychics, astrology etc. What value does the concept of anticipated experience have if it fails to filter out even the most common fallacious beliefs?
4Vladimir_Nesov
That there are books about ghosts is evidence for ghosts existing (but also for lots of other things). There are also arguments against this hypothesis, both a priori and observational. A good model/theory also explains why you'd read about ghosts even though there is no such thing.
0Dpar
You're not addressing my core point though. If the criteria of anticipated experience as you define it is as likely to be satisfied by fallacious beliefs as it is by valid ones, what purpose does it serve?
2Vladimir_Nesov
I addressed that question in this comment; if something is unclear, ask away. The difference is between a belief that is incorrect, and a belief that is not even wrong.
2Dpar
Alright, I think I see what you're getting it, but I still can't help but think that your definition of sensory experience is too broad to be really useful. I mean the only type of belief that it seems to filter out is absolute nonsense like "I have a third leg that I can never see or feel", did I get that about right?
2Vladimir_Nesov
Yes. It happens all the time. It's one way nonsense protects itself, to persist for a long time in minds of individual people and cultures. (More generally, see anti-epistemology.)
2Dpar
So essentially what you and Eliezer are referring to as "anticipated experience" is just basic falsifiability then?
6Vladimir_Nesov
With a bayesian twist: things don't actually get falsified, don't become wrong with absolute certainty, rather observations can adjust your level of belief.
3SilasBarta
Slightly OT, but this relates to something that really bugs me. People often bring up the importance of statistical analysis and the possibility of flukes/lab error, in order to prove that, "Popper was totally wrong, we get to completely ignore him and this out-dated, long-refuted notion of falsifiability." But the way I see it, this doesn't refute Popper, or the notion of falsifiability: it just means we've generalized the notion to probabilistic cases, instead of just the binary categorization of "unfalsified" vs. "falsified". This seems like an extension of Popper/falsifiability rather than a refutation of it. Go fig.
4Vladimir_Nesov
I reached much clearer understanding once I've peeled away the structure of probability measure and got down to mathematically crisp events on sample spaces (classes of possible worlds). From this perspective, there are falsifiable concepts, but they usually don't constitute useful statements, so we work with the ones that can't be completely falsified, even though parts of them (some of the possible worlds included in them) do get falsified all the time, when you observe something.
0[anonymous]
Isn't that like saying we've generalized the theory that "all is fire" to cases where the universe is only part fire? If falsification is absolute then Popper's insight that "all is falsification" is just plain wrong; if falsification is probabilistic then surely the relevant ideas existed before Popper as probability theory. It's not like Popper invented the notion that if a hypothesis is falsified we shouldn't believe it.
7Dpar
Ok, I understand what you mean now. Now that you've clarified what Eliezer meant by anticipated experience my original objection to it is no longer applicable. Thank you for an interesting and thought provoking discussion.
1jimrandomh
Falsifiability can be quantified, in bits. If the only test you have for whether something's true or not is something lame like whether it appears in stories or not, then you have a tiny amount of falsifiability. If there is a large supply of experiments you can do, each of which provides good evidence, then it has lots of falsifiability. (This really deserves to be formalized, in terms of something along the lines of expected bits of net evidence, but I'm not sure how to do so, exactly. Expected bits of evidence does not work, because of scenarios where there is a small chance of lots of evidence being available, but a large chance of no evidence being available.)
4SilasBarta
Just a note about terminology: "expected bits of evidence" also goes by the name of entropy, and is a good thing to maximize in designing an experiment. (My previous comment on the issue.) And if I understand you correctly, you're saying that the problem with entropy as a measure of falsifiability, is that someone can come up with a crank theory that gives the same predictions in every single case, except one that is near impossible to observe, but which, if it happened, would completely vindicate them? If so, the problem with such theories is that they have to provide a lot of bits to specify that improbable event, which would be penalized under the MML formalism because it lengthens the hypothesis significantly. That may be want you want to work into a measure of falsifiability. But then, at that point, I'm not sure if you're measuring falsifiability per se, or just general "epistemic goodness". It's okay to have those characteristics you want as a separate desideratum from falsifiability.
2Dpar
Isn't it an essential criteria of falsifiability to be able to design an experiment that can DEFINITIVELY prove the theory false?
7RobinZ
That is the criterion which the Bayesian idea of evidence lets you relax. Instead of saying that "you need to be able to define experiments where at least one result would be completely impossible by the theory", a Bayesian will tell you that "you need to be able to define experiments where the probability of one result under the theory is significantly different from the probability of another result". Look at, say, the theory that a coin is weighted towards heads. If you want to be pedantic, no result can "definitely prove" that it is not (unusual events can happen), but an even split of heads and tails (or a weighting towards tails) is much more unusual given that theory than a weighting towards heads. Edit PS: I am totally stealing the meme that "Bayes is a generalization of Popper" from SilasBarta.
1SilasBarta
Steal the meme, and spread it as far and as wide as you possibly can! The sooner it beats out "Popper is so 70 years ago", the better. (Kind of ironic that Bayes long predated Popper, though the formalization of [what we now call] Bayesian inference did not.) Example of my academically-respected arch-nemesis arguing the exact anti-falsificationist view I was criticizing.
3thomblake
I'm pretty sure that was handily discussed in An Intuitive Explanation of Bayes's Theorem and A Technical Explanation of Technical Explanation.
0RobinZ
Ehhcks-cellent!
7SilasBarta
Fair point, and it was EY's essay that showed me the connection. But keep in mind, the point of the essay is, "Bayesian inference is right, look how Popper is a crippled version of it." My point in saying "my" meme is different: "Popper and falsificationism are on the right track -- don't shy away from the concepts entirely just because they're not sufficiently general." It's a warning against taking the failures of Popper to mean that any version of falsificationism is severely flawed.
5JoshuaZ
As Robin's explained below Bayesianism doesn't do that. You should also see the works of Lakatos and Quine where they discuss the idea that falsification is flawed because all claims have auxiliary hypotheses and one can't falsify any hypothesis in isolation even if you are trying to construct a neo-Popperian framework.
4SilasBarta
Yes, but that still doesn't show falsificationism to be wrong, as opposed to "narrow" or "insufficiently generalized". Lakatos and Quine have also failed to show how it's a problem that you can't rigidly falsifiy a hypothesis in isolation: Just as you can generalize Popper's binary "falsified vs. unfalsified" to probabilistic cases, you can construct a Bayes net that shows how your various beliefs (including the auxiliary hypotheses) imply particular observations. The relative likelihoods they place on the observations allow you to know the relative amount by which those various beliefs are attenuated or amplified by any particular observation. This method gives you the functional equivalent of testing hypotheses in isolation, since some of them will be attenuated the most.
2JoshuaZ
Right, I was speaking in a non-Bayesian context.
1satt
If I remember rightly, that's where poor old Popper came unstuck: having thought of the falsifiability criterion, he couldn't work out how to rigorously make it flexible. And as no experiment's exactly 100% uppercase-D Definitive, that led to some philosophers piling on the idea of falsifiability, as JoshuaZ said. But more recent work in philosophy of science suggests a more sophisticated way to talk about how falsifiability can work in the real world. The key idea is "severe testing", where a "severe test" is a test likely to expose a specific error in a model, if such an error is present. Those models that pass more, and more severe, tests can be regarded as more useful than those that don't. This approach also disarms the "auxiliary hypotheses" objection JoshuaZ paraphrased; one can just submit those hypotheses to severe testing too. (I wouldn't be surprised to find out that's roughly equivalent to the Bayes net approach SilasBarta mentioned.)
2anon895
I was expecting the link to be Mundane Magic.
0Vladimir_Nesov
The point is not that the ability is "magical", but that it's real, that we do have an ability to read minds, in exactly the same sense as Dpar appealed to the impossibility of.
3RobinZ
Belatedly: Welcome to Less Wrong! Please feel free to introduce yourself.
0Dpar
A belated thanks! :) DP
2MarsColony_in10years
The LessWrong FAQ says that there is value in replying to old content, so I'm commenting in hopes that it is useful to someone in the future, and just for the sake of organizing my thoughts. I would have phrased this differently than Yudkowsky, but I think I understand the concept he was getting at when he gave this example: His point is that this is just semantics. It makes no difference to the world whether we label something "post-utopian" or "aegffsdfa eereraksrfa" or anything else. The words you read in the book will be the same. The reason I don’t like this example is that, if I actually knew some literary jargon, I might get some real verifiable information that does actually mean I should expect a specific kind of sensory experience. It’s just that the classification scheme is arbitrary, and so is my belief that one classification scheme is "correct". The label is just a label, so arguing about classification schemes is just semantics. Using this definition, your belief that the crusades took place would affect what sorts of things you would expect to read, and what sorts of archeological finds you would expect to find if you went looking for them. However, if you believe that the crusades marked the beginning of the high middle ages, that would just be semantics. We could say that the middle ages started at the sacking of Rome, or we could make a label like "dark ages" to describe the intermediary period. What we call it and how we classify it makes no difference in the actual reality of history. It's just semantics.
4TheAncientGeek
Semantic labels are part of the structure of an explicit model. For instance, the Chinese use the same word for both "rat" and "mouse". A model with a ratmouse vertex will behave differently to a model with separate rat and mouse verteces. The structure and function of model affect what it predicts, what it's users can notice, how they behave. Agents do not passively receive a stream of predetermined experiences, they interact with the world, and the experiences they can expect depend on the structure and function of their models... ..and more besides. Models contain evaluative weightings as well as neutral structure. For instance, in the English speaking world, mice have the connotation of being cute, rats of being vermin. The professor might not be failing to specify an empirical confirmable concept when describing the writer as a post utopian: she might rather be succeeding in tweaking her students' evaluative model. She might be aiming at making a social or political point. There is a long history of the political influence of language ranging from Greek rhetoricIan's to Orwell' s essays. A STEM type might consider it pointless, to focus on such issues, rather than what can be proved objectively. A humanities type might also consider it pointless to focus on objective, empirical claims with no social or political upshot. Neither complaint is really about meaningfullness or semantics, in the sense if the meaningfulness of the words, rather they are both about the subjectively evaluated pointfulness of an activity. By a convoluted meta level irony, the way the way the term "semantics" is often used is itself a way if funneling the reader towards a conclusion. We have seen that there are circumstances where a semantic change would make a difference: where it makes a structural/functional change, and where it makes an evaluative/connotational difference. Since these circumstances don't always to apply, there are circumstances where a semantic change really is tri
0MarsColony_in10years
Thanks for breaching that topic. I considered pointing out that my "aegffsdfa eereraksrfa" example might be more difficult to pronounce than "post-utopian", and so actually would have an impact on the world in general. On reflection, I decided to make the assertion that it "makes no difference", since that would spare a lot of confusion. It's a good first order approximation. When introducing a topic, it's important to take the Bohr model view of the world before trying to explain quarks and leptons. The entanglement of semantic language with our interpretation of reality clouds things. Scientific language is precise, but often dry and hard to understand. However, by de-coupling the two worlds, we study the underlying reality without those (or perhaps with only minimal) distorting effects from our language. That's what we are doing when we talk about Map and Territory here on LW. We get a better map from this, but if we also compare the collective maps of societies to the best maps of reality, we can look for systematic differences. Some of these are cognitive biases, which we tend to concentrate on here on LW. However, there are also many other interesting or useful things that we can learn about ourselves as mapmakers. For example, the Bouba/kiki effect might help us choose more intuitive vocabulary as we build a more and more extensive set of jargon. Just studying the way languages evolve can be informative, whether it's rigorously using Computational Linguistics or informally by an author or artist. The mere existence of a formal scientific understanding of reality allows a poet or philosopher, if they are familiar only with the answers but not the underlying explanations, to look at some facet of human nature and ask "isn't it odd when people...". A great deal of social commentary is built from that one question.

You write, “suppose your postmodern English professor teaches you that the famous writer Wulky Wilkinsen is actually a ‘post-utopian’. What does this mean you should expect from his books? Nothing.”

I’m sympathetic to your general argument in this article, but this particular jibe is overstating your case.

There may be nothing particularly profound in the idea of ‘post-utopianism’, but it’s not meaningless. Let me see if I can persuade you.

Utopianism is the belief that an ideal society (or at least one that's much better than ours) can be constructed, for example by the application of a particular political ideology. It’s an idea that has been considered and criticized here on LessWrong. Utopian fiction explores this belief, often by portraying such an ideal society, or the process that leads to one. In utopian fiction one expects to see characters who are perfectible, conflicts resolved successfully or peacefully, and some kind of argument in favour of utopianism. Post-utopian fiction is written in reaction to this, from a skeptical or critical viewpoint about the perfectibility of people and the possibility of improving society. One expects to see irretrievably flawed characters, i... (read more)

Would you consider Le Guin's The Dispossessed to be post-utopian? I think she intends her Anarres to be a good place on the whole, and a decent partial attempt at achieving a utopia, but still to have plausible problems.

3tog
Not to go off on a tangent, but I'd say it's more utopian than critical of utopia - I don't think we can require utopias to be perfect to deserve the name, and Anarres is pretty (perhaps unrealistically) good, with radical (though not complete) changes in human nature for the better.
2Jack
Brave New World is definitely dystopian, not post-utopian. Nancy's suggestion for post-utopian is exactly right. I definitely agree that we can meaningfully classify cultural production, though.

I think it's both. "Brave New World" portrays a dystopia (Huxley called it a "negative utopia") but it's also post-utopian because it displays skepticism towards utopian ideals (Huxley wrote it in reaction to H. G. Wells' "Men Like Gods").

I don't claim any expertise on this subject: in fact, I hadn't heard of post-utopianism at all until I read the word in this article. It just seemed to me to be overstating the case to claim that a term like this is meaningless. Vague, certainly. Not very profound, yes. But meaningless, no.

The meaning is easily deducible: in the history of ideas "post-" is often used to mean "after; in consequence of; in reaction to" (and "utopian" is straightforward). I checked my understanding by searching Google Scholar and Books: there seems to be only one book on the subject (The post-utopian imagination: American culture in the long 1950s by M. Keith Booker) but from reading the preview it seems to be using the word in the way that I described above.

The fact that the literature on the subject is small makes post-utopianism an easier target for this kind of attack: few people are likely to be familiar with the idea, or motivated to defend it, and it's harder to establish what the consensus on the subject is. By contrast, imagine trying to claim that "hard science fiction" was a meaningless term.

Indeed. Some rationalists have a fondness for using straw postmodernists to illustrate irrationality. (Note that Alan Sokal deliberately chose a very poor journal, not even peer-reviewed, to send his fake paper to.) It's really not all incomprehensible Frenchmen. While there may be a small number of postmodernists who literally do not believe objective reality exists, and some more who try to deconstruct actual science and not just the scientists doing it, it remains the case that the human cultural realm is inherently squishy and much more relative than people commonly assume, and postmodernism is a useful critical technique to get through the layers of obfuscation motivating many human cultural activities. Any writer of fiction who is any good, for instance, needs to know postmodernist techniques, whether they call them that or not.

6TheOtherDave
Yes. That said, it's not too surprising that postmodernists are often the straw opponent of choice. The idea that the categories we experience as "in the world" are actually in our heads is something postmodernists share with cognitive scientists; many of the topics discussed here (especially those explicitly concerned with cognitive bias) are part of that same enterprise. I suspect this leads to a kind of uncanny valley effect, where something similar-but-different creates more revulsion than something genuinely opposed would. Of course, knowing that does not make me any less frustrated with the sort of soi-disant postmodernist for whom category deconstruction is just a verbal formula, rather than the end result of actual thought. I also weakly suspect that postmodernists get a particularly bad rap simply because of the oxymoronic name.
4David_Gerard
Oh yeah. While it's far from a worthless field, and straw postmodernists are a sign of lazy thinking, it is also the case that postmodernism contains staggering quantities of complete BS. Thankfully, these are also susceptible to postmodernist analysis, if not by those who wish to keep their status ...
-1BarbaraB
I played a mental game trying to make predictions based on the information, that Wulky Wilkinsen is post-utopian and shows colonial allienation - never heard of any of that before :-). Wulky Wilkinsen is post-utopian ... I expect to find a bunch of critically acclaimed authors, who wrote their most famous books before Wulky wrote his most famous books (5 - 15 years ahead ?), lived in the same general area as Wulky, and portrayed people who were more altruistic and prone to serve general good than we normally see in real life. It does not say too much about the actual writing style of Wulky - he could have written either in the similar way as "the bunch" (utopians), or just the opposite - he could have been just fed up by the utopians' style and portray people more evil than we normally see in everyday life. So my prediction does not tell what Wulky's books feel like, but it is still a prediction, right ? Colonial allienation - the book contains characters that have lived in a colony (e.g. India) for a long time (athough they might have just arrived to the "maternal" colonial country, e.g. Britain). These characters are confronted with other characters that have lived in the "maternal" colonial country for a long time (athough they might have just arrived to the colony :-) ). There are conflicts between these two groups of people, based on their background. They have different preferences when they are making decisions, probably involving other people. Thus they are allienated. Do not tell me this was not the point of Eliezer's post, let me just have some fun !

How is this not just a simple arguement on semantics (on which I believe a vast majority of arguements are based)?

They both accept that the tree causes vibrations in the air as it falls, and they both accept that no human ear will ever hear it. The arguement appears to be based solely on the definition, and surrounding implications, of the word "sound" (or "noise" as it becomes in the article) - and is therefore no arguement at all.

3bigjeff5
I think that may have been the point: You can define a thing based on any criteria you like. It simply has to allow your expectations to agree with reality in order for it to be true. One says "it is sound because it vibrates regardless of whether anyone hears it." This person believes that sound is the vibrations. The other says "it is not sound because it is never processed in a mind." This person does not deny that the vibrations exist, he simply believes it isn't sound until someone hears it. These two have different definitions of "sound", but within their definitions both allow expectations that are completely consistent with reality. The point is to make sure your beliefs "pay rent" - that they allow you to have expectations that match up with reality. If the second person had the same belief of what sound was as the first (i.e. vibrations in the air), yet also believed that vibrations in the air do not occur when there is nobody to hear them, that belief would not pay rent. When they recorded the sound with nobody around he would expect there to be nothing at all on the tape, yet there would be something on the tape. The only way to resolve this is to adjust your belief after the fact, which means your belief couldn't pay its rent.