Open Thread: September 2009

I declare this Open Thread open for discussion of Less Wrong topics that have not appeared in recent posts.

179 comments, sorted by
magical algorithm
Highlighting new comments since Today at 10:33 PM
Select new highlight date

In light of how important it is (especially in terms of all the decision theory posts!) to know about Judea Pearl's causality ideas, I thought I might share something that helped me get up to speed on it: this lecture on Reasoning with Cause and Effect.

For me, it has the right combination of detail and brevity. Other books and papers on Pearlean causality were either too pedantic or too vague about the details, but I learned a lot from the slides, which come with good notes. Anyone know what proceedings papers it refers to though?

Hm, on first browse this does seem like a good intro.

Does anyone feel the occasional temptation to become religious?

I do, sometimes, and push it away each time. I doubt that I could, really, fall for that temptation - even if I tried to, the illogicality of the whole thing would very likely prevent me from really believing in even a part of it very seriously. And as more and more time passes, religious structures begin to seem more and more ridiculous and contrived. Not that I'd have believed that them starting to feel more contrived would even have been possible.

And yet... occasionally I remember the time back in my teens, when I had some sort of a faith. I remember the feeling of ultimate safety it brought with it - the knowledge that no matter what happens, everything will turn out well in the end. It might be a good thing that I spend time worrying over existential risks, and spend time thinking about what I could do about them, but it sure doesn't exactly improve my mental health. The thought of returning to the mindset of a believer appeals on an emotional level, in the same way stressed adults might longingly remember the carefree days of childhood. But while you can't become a child again, becoming a believer is at least theoretically possible. And sometimes I do play around with the idea of what it'd be like, to adopt a belief again.

Uh, here is a confession. Twice in the last 6-7 years, at moments of extreme psychological distress, I talked to the God of my Catholic youth. Once I went to an empty church after a series of coincidences (running into two people from my Catholic grade school separately) that I thought was a sign from God. Really, embarrassing, right? It was like my mind segmented and the rationalist was put aside and the devout Catholic school boy was put in charge. The last time this happened was about four years ago.

I don't know if this could still happen today (my atheism is probably more entrenched now). There have been plenty of periods of distress where this didn't happen, so I don't know what triggered it in particular. I think my brain must have really needed a God figure at that moment and didn't know how to deal with the pain without one so it hacked itself and turned off the rationalist defenses. Or something, it seems so screwed up looking back on it.

Interestingly, I told my theistic-non religious girlfriend about this who in turn told her Christian best friend. Talking to me on the phone for the first time, the friend something along the lines of "I know you say you're an atheist but B(my girlfriend) told me that you sometimes pray when you're upset so I know you're really a good person/God loves you." In other words, I'm least wrong when I'm thinking least clearly.

On the consolidation of dust specks and the preservation of utilitarian conclusions:

Suppose that you were going to live for at least 3^^^3 seconds. (If you claim that you cannot usefully imagine a lifespan of 3^^^3 seconds or greater, I must insist that you concede that you also cannot usefully imagine a group of 3^^^3 persons. After all, persons are a good deal more complicated than seconds, and you have experienced more seconds than people.)

Suppose that while you are contemplating how to spend your 3^^^3-plus seconds, you are presented with a binary choice: you may spend the next 50 years of this period of time being tortured, or you may spend the next 3^^^3 seconds with a speck of dust in your eye that you cannot get rid of until that time period is up. (Should you succeed in uploading or similar over the course of the next 3^^^3 seconds, the sensation of the speck in the eye will accompany you in the absence of a physical eye until you have waited it out). Assume that after the conclusion of the torture (should you select it), you will be in fine physical health to go on with the rest of your lengthy life, although no guarantees are made for your sanity. Assume that the speck of dust does not impede your vision, and that you will not claw out your eye trying to be rid of it at any time; likewise, no guarantees are made for your sanity.

What selection would you make?

If this choice was actually presented to someone, my guess would be that he would first choose the speck, and then after an extremely long time (i.e. much, much longer than 50 years, giving him a sense of proportion) he would undergo a preference reversal and ask for the torture.

What if the TORTURE occurs during a random time in the next 3^^^3 seconds, not right at the beginning? Also, I think we definitely require a limit on sanity damage because otherwise the scenario is being tortured for 50 years and then spending the next 3^^^3 seconds being insane which Vastly outweighs the ordinary scenario of being tortured for 50 years.

In the original scenario, where just some random person got tortured, no constraints were specified about eir sanity or lifespan post-torture.

I think I did specify that no one would die who would otherwise be immortal; eternal insanity or 3^^^3 years of insanity ought to be implicitly included, I'd think.

although no guarantees are made for your sanity

This phrase makes the difference for me- the 3^^^3 other people in the original argument weren't mad- or at least, no more than would have been mad anyway.

Additionally, in your scenario, we have to consider discount rates- it's certainly conceivable that someone might choose the dust specks over torture now, but be willing to forgo the dust specks in return for torture in 3^^^3 seconds time.

Does it seem likely to you that out of 3^^^3 people chosen with no particular safeguards, not one of them will find a dust speck in the eye to be maddening? It could be the last straw in a string of misfortunes; it could set off some causal chain that will lead to other maddening events, etc.

3^^^3 is such a huge number, some must find it maddening, but the proportion will be a lot lower than the odds that 50 years of torture breaks you mentally.

That's interesting: I have much, much less hesitation in saying TORTURE to this one. With the original, I can grudgingly concede that I suppose I possibly ought to choose TORTURE, but I still can't ever quite convince myself to feel that it's a good answer. This one, I think I can.

Suppose you are given a button that you can press at any time during the 50 years of torture, that will stop the torture (and erase your memory of it if you wish), but you'll have to live with the dust speck from then on.

I predict that you'll press the button after actually being tortured for a couple of hours, maybe days, but at most weeks. Even professional spies/soldiers/terrorists who have trained to resist torture end up betraying their cause, so I find it hard to believe that you can hold out for 50 years.

But if you really prefer TORTURE now, that brings up an interesting question: whose preferences are more important, the current you, or the hypothetical future you? It could be argued that the future you is in a better position to decide, since she knows what it actually feels like to be tortured for a significant period of time, whereas you don't.

But I don't consider that a knock-down argument, so what do you think? Suppose you can also commit to not pressing the button (say by disabling your arm/hand muscles for 50 years), would you do so?

(This is related to a recent comment by Rolf Andreassen, which I think applies better to this scenario.)

Yes, I think you're almost certainly right about the button, which thought does indeed put a dent in my lesser hesitation in choosing TORTURE. I think I would definitely not commit to not pressing the button if I were able to "try out" the SPECKS scenario for some short period of time first (say, a week). That way I could make a comparison. Absent that condition... I don't know. I can't imagine having the guts to commit to not pressing it.

The fact is that while I certainly don't see a momentary dust speck as torture, I can easily imagine beginning to see it as torture after a day, never mind 3^^^3 seconds, which is rather longer than 50 years. But I can't be certain of that, nor of to what extent I would get used to it, nor of how it would compare with much worse kinds of torture. (But then again, there's a spanner in the works: knowing that you won't have any lasting physical damage... perhaps that would lend some kind of strength of mind to a torturee?)

Sean Carroll and Carl Zimmer are leaving Bloggingheads, mostly because it's started playing nice with creationists. Click their names to read their full explanations.

Thanks for the link. I saw the original Behe podcast and was surprised when it suddenly disappeared with a brief note from an admin. I haven't been able to follow up on this matter much since it hasn't generated much buzz, but I'll look forward to reading these links - if people are boycotting Bloggingheads for giving platforms to people whom they don't like.... well, that's an interesting precedent to set which should be scrutinized carefully.

Hm. (Reads.) Well... if BHTV has me on, an "anti-accomodationist" as I've recently heard my diavlogs called (what a lovely term!), then I don't think it's unfair for them to give Behe a slot. It is extremely foolish that they paired him up with a linguist. (Reads.) Looks like Robert Wright agrees with that last part.

You're one of the last people I would have expected to be concerned with 'fairness' when one of the sides is blatant crackpottery. I suppose you wouldn't have a problem if BHTV invited an astrologer, as long as they paired him up with an astronomer?

What harm is done by bringing on an astrologer? At worst it fail to amuse.

But it's obvious you're not talking about the diavlog's impact on you... you're concerned with the poor, unwashed masses who might actually be left to form their own opinions from the available information. Well, that's very nice of you, but I don't think it's unreasonable to believe that it's safe to expose people to views which might be labeled as "crackpottery" by some.

High-status serious people want to associate with high-status others. Allowing crackpots on the same venue dramatically decreases its attractiveness for quality participants.

You can't really give an elaborate justification for why in this particular case it's OK, because signals are shallow.

The "unwashed masses", as you call them, are already getting plenty of exposition to crackpottery, much more than they get to real science, so that a few crackpots on BHTV are barely a drop in the ocean. That's not what concerns me, no.

What concerns me is that BHTV has a reputation as a respectable website thanks to the participation of respectable academics and experts. It's reasonable to assume that such a respectable website wouldn't invite crackpots to promote their brand of crackpottery; in fact, that's an assumption I made myself until I read Sean's and Carl's posts. Inviting crackpots therefore gives the impression that these people should be taken seriously, even if we think they're wrong.

In any event, your own motives are suspect, to say the least. Characterizing creationist nuts as "people whom [Sean and Carl] don't like", as if creationism was merely a distasteful political opinion, or something, makes you sound like a crackpot yourself, or worse, a postmodernist.

So BHTV can't both enjoy the participation of respectable academics and also host the occasional crackpot? There exists no such universe where the two could possibly coincide? Is there some implicit assumption here that there's a fixed amount of BHTV episodes, each of which will feature either crackpots or respectable academics? Even if this were so, wouldn't the reasonable response be to skip over the crackpots rather than avoiding the entire medium? The only justifiable rationale I can see for skipping over BHTV because of this is if you just watched diavlogs at random and having crackpots degraded the signal:noise ratio of the site. But I doubt that you, I, Sean, Carl, or your average Bloggingheads viewer navigates the site in this manner.

Even though I profoundly degree with Behe's epistemology (and theology), which should go without saying in these parts, I found the debate interesting (I think irreducible complexity is a neat topic), certainly moreso than I've enjoyed other diavlogs. Can anyone honestly say that Behe's presence is less valuable than any other podcast on the website? I doubt it, and thus it strikes me as disingenuous that the unique response his presence generates can be explained away purely through outrage at the notion that somewhere, someone's time may be wasted.

And I didn't include the "people whom others don't like" line as a defense of creationism per se, but as a broader point about silencing views found in contempt. To rip off Will Wilkinson, I'd probably venture to assert that unrepentant Marxists are just as high on the crackpottery scale as Creationists, but I highly doubt we'd see people abandon the site in protest in BHTV hosted some of them. "Respectability" in this context is a tricky term to use, since "respectability" tends to be conferred by social fashions just as much as actual correspondence to whatever virtues we've deemed to be worthy of respect. On a more base level, I suspect that many participants in this community have been dismissed as "crackpots" in some context or other before, and are skeptical of the neutrality and intellectual virtues of those who tend to yield the power of the censor. This isn't a philosophical defense of subjectivism or postmodernism, but an institutional defense of the rough reasons why we don't just go ahead and burn Behe at the stake.

Should BHTV invite Perez Hilton to debate the fearsome Man Bat? Michael Behe is as credible an author as Pamela Anderson, although not quite as illuminating. I used to think that the worst kind of ignorance was when you knew you were wrong and refused to accept it. Now I think the worst kind is when you know you're capable of knowing when you're wrong but refuse to let yourself. Michael Behe wants to be ignorant of his own ignorance. Let him do so in the peace and quiet of his own sad little world.

You're shifting the goalposts some. I'm not defending the original decision to invite Behe. I'm questioning the notion that inviting Behe is such an egregious offense against BHTV's "respectability" that it should be boycotted. I wouldn't boycott BHTV if 90% of the diavlogs were replaced by midget porn, if it meant that I would get the occasional episode of Free Will.

I think Behe's critics should just admit that what's really motivating the reaction is the notion that Creationists not only should not be given forums to speak, but those who do grant Creationists forums to speak should be actively identified and boycotted in a way which is reserved for an arguably arbitrarily-defined set of social undesirables. This isn't an indefensible position, but people have to admit to holding this belief (or some similar belief which is constructed in a more-charitable manner) before a meaningful debate can be enjoined.

[Edit]

Reading over the comments section of the CV posts, it looks like a lot of people are quick to point to Megan McArdle as the political crackpot equivalent of Behe. Should her presence be boycotted too as detrimental to the site? Where should the line be drawn? Where do you actually think the line would be drawn, if not along questionable ideological lines? Why have a line at all?

I think Behe's critics should just admit that what's really motivating the reaction is the notion that Creationists not only should not be given forums to speak, but those who do grant Creationists forums to speak should be actively identified and boycotted in a way which is reserved for an arguably arbitrarily-defined set of social undesirables.

What critic will not admit that? It's hardly a fringe opinion in the scientific community that Creationists should not be given forums to speak on the thoroughly unscientific topic of Creationism, and that those who do so and call it science are being absurdly and unnecessarily tolerant. Creationism has never been more or less than an attack on science. It's extremely toxic, and while I would never try to "silence" anyone, I don't think it deserves more publicity. I grew up being taught that dreck in a fundamentalist Christian school and I'm more familiar with Behe than I'd care to be. Frankly, he's an idiot, and his life purpose seems to be toward making more idiots. He doesn't need anyone's help.

As for McArdle, I don't really care. Politics is not a hard science, and while she's something of a crackpot, she's not that way because somebody proved her map doesn't follow the territory. It's the difference between someone who thinks the earth is flat and someone who thinks it's run by the Illuminati. The former is just wrong, the latter is just crazy. I don't mind crazy, because crazy isn't nearly as dangerous as wrong.

I haven't seen BhTV endorse Creationism as science in any official capacity.

might be labeled as 'crackpottery' by some.

We been known to shoot us some subjectivists 'round these here parts, y'hear?

Sean Carrol concurs

What I objected to about the creationists was that they were not worthy opponents with whom I disagree; they're just crackpots. Go to a biology conference, read a biology journal, spend time in a biology department; nobody is arguing about the possibility that an ill-specified supernatural "designer" is interfering at whim with the course of evolution. It's not a serious idea. It may be out there in the public sphere as an idea that garners attention -- but, as we all know, that holds true for all sorts of non-serious ideas.

It seems like science snobbery. BHTV has loads of political commentary - and other non-science. It just isn't remotely like a peer-reviewed science journal.

He says 48 minutes in that the Templeton foundation "were no longer" and that they only supported the site for four months.

However, their ads are still on the sponsored videos beneath:

http://bloggingheads.tv/percontations

...so the Templeton foundation link apparently remains. Maybe there will be no more percontations, though.

Wright gives much the same arguments about ID as I give - in the section on "Viable Intelligent Design Hypotheses" - on:

http://originoflife.net/intelligent_design/

Readers of Less Wrong may be interested in this New Scientist article by Noel Sharkey, titled Why AI is a dangerous dream, in which he attacks Kurzweil's and Moravec's "fairy tale" predictions and questions whether intelligence is computational ("[the mind] could be a physical system that cannot be recreated by a computer").

[edit] I thought this would go without saying, but I suspect the downvotes speak otherwise, so: I strongly disagree with the content of this article. I still consider it interesting because it is useful to be aware of differing and potentially popular perspectives on these subjects (and Sharkey is something of a "populist" scientist). I think the opinions it espouses are staggeringly ill-conceived, however.

I strongly disagree. First, on the grounds that LW readers have strong reason to believe this:

("[the mind] could be a physical system that cannot be recreated by a computer").

to be false, and so treat it similarly to a proof that 2=1.

But instead of just being a grouch this time, I decided to save you guys the effort and read it myself to see if there's anything worth reading.

There isn't. It's just repetitions of skepticism you've already heard, based on Sharkey's rejection of "the assumption that intelligence is computational" (as opposed to what, and which is different and uncreatable why?), which "It might be, and equally it might not be".

Other than that, it's a puff piece interview without much content.

(Phase 1)

Agreed, I don't see why the mind isn't a type of "computer", and why living organisms aren't "machines". If there was something truly different and special about being organic, then we could just build an organic AI. I don't get the distinction being made.

(Phase 2)

technological artifacts that have no possibility of empathy, compassion or understanding.

Oh: sounds like dualism of some kind if it is impossible for a machine to have empathy, compassion or understanding. Meaning beings with these qualities are more than physical machines, somehow.

(Phase 3)

Reading through some of the comments to the article, it sounds like the objection isn't that intelligence is necessarily non-physical, but that "computation" doesn't encompass all possible physical activity. I guess the idea is that if reality is continuous, then there could be some kind of complexity gap between discrete computation and an organic process.

Phases 1-3 is the sequential steps I've taken to try to understand this point of view. A view can't be rejected until its understood...I'm sure people here have considered the AI-is-impossible view before, but I hadn't.

What is the physical materialist view on whether reality is discrete? (I would guess it's agnostic.) What is the AI view on whether computations must be discrete? (I would guess AI researchers wouldn't eschew a continuous computation as as a non-computational thing if it were possible?)

I agree it's important to apply the principle of charity, but people have to apply the principle of effort too. If Sharkey's point is about some crucial threshold that continuous systems possess, he should say so. The term "computational" is already taken, so he needs to find another term.

And he can't be excused on the grounds that "it's a short interview", considering that he repeated the same point several times and seemed to find enough space to spell out what (he thinks) his view implies.

"[the mind] could be a physical system that cannot be recreated by a computer"

Let me quote an argument in favor of this, despite the apparently near universal consensus here that it is wrong.

There is a school of thought that says, OK, let's suppose the mind is a computation, but it is an unsolved problem in philosophy how to determine whether a given physical system implements a given computation. In fact there is even an argument that a clock implements every computation, and it has yet to be conclusively refuted.

If the connection between physical systems and computation is intrinsically uncertain, then we can never say with certainty that two physical systems implement the same computation. In particular, we can never know that a given computer program implements the same computation as a given brain.

Therefore we cannot, in principle, recreate a mind on a computer; at least, not reliably. We can guess that it seems pretty close, but we can never know.

If LessWrongers have solved the problem of determining what counts as instantiating a computation, I'd like to hear more.

If LessWrongers have solved the problem of determining what counts as instantiating a computation, I'd like to hear more.

Sure thing. I solved the problem here and here in response to Paul Almond's essays on the issue. So did Gary Drescher, who said essentially the same thing in pages 51 through 59 of Good and Real. (I assume you have a copy of it; if not, don't privately message me and ask me how to pirate it. That's just wrong, dude. On so many levels.)

This was linked on Hacker News http://news.ycombinator.com/item?id=797871

I left this comment there: I thought this might be something like Eliezer's arguments against developing a GAI until it could be made provably Friendly AI, instead I just got an argument exactly like the ones in 1903 that said heavier than air flight by men was impossible - go back and read some of them, some of the arguments were almost identical. Some of the arguments are currently true, but some of them amount to "I can't do it, and no one else has done it, therefore there must be some fundamental reason it can't be done".

Since the topic of atheism, morality, and the like often come up here, I would like to point people to the free online book Secular Wholeness by David Cortesi. He approaches the topic of religion by trying to determine what benefits it can provide (community, challenges to improve oneself, easier ethical decisions, etc.), then tries to describe how to achieve these same benefits without resorting to religion. It's not very heavy on detail but seems very well sourced and with some good pointers on why people choose religions and what they get out of them.

ETA: This could have been a reply to the thread on Scientology, had I seen it before posting.

It's been awhile since this came up. But hay, look, arguments about sex and race and stuff, as applicable to atheism as to rationality in general.

Boo to whoever voted this down in the first place. Great link.

It's obviously a counter-conspiracy against my conspiracy to get people to see rationality as something that exists outside of this site. And also to make you look at girls.

Is there an easy way to access the first comment of someone without looking at their comment page and uploading "next" zillions of times?

You're probably wondering why I would want to do that. I have been motivated occasionally to read someone's comments from beginning to end, and today I found myself wondering what my first comment was about.

My mind frequently returns to and develops the idea that sometime in the future, a friendly artificial intelligence is going to read Less Wrong. It uploads all the threads and is simultaneously able to (a) read all the comments as they were written chronologically, and (b) keep track of each person's comments as they chronologically developed and evolved. I probably fantasize about this because I like to analyze patterns and wish I could do that.

The AI would notice all sorts of patterns that I only scantly glimpse...

  • patterns regarding time of day and frequency of postings; building posteriors about where people live (Europe or the U.S.) and whether they're a night owl or an early riser or travel across the Atlantic frequently

  • patterns of speech and the mutual social evolution of these patterns among LWers, yet still being able to tell if people communicate with one another outside LW or if they're really multiple accounts for the same person

  • of course, this FAI notices that byrnema has most excellent qualities, etc

What else would the FAI learn?

Does anyone else have this daydream?

Does anyone else have this daydream?

Earlier today I was assigning probabilities to whether or not someone on this site was someone I used to date. If they aren't then I really should introduce them...

Scott Aaronson announced Worldview Manager, "a program that attempts to help users uncover hidden inconsistencies in their personal beliefs".

You can experiment with it here. The initial topics are Complexity Theory, Strong AI, Axiom of Choice, Quantum Computing, Libertarianism, Quantum Mechanics.

I just tried the one for AI and I think its not quite accurate. One of the biggest issues is that I think some of the terms need to be precisely defined and they are not. The other issue I found was that the analysis of my beliefs was not completely accurate because it did not take into account all the answers properly.

Its an interesting idea but needs work.

Mostly agree is a higher degree of agreement than Agree ?

To Somewhat agree that everyone should have the vote and Disagree that children should have the vote is inconsistent ?

Obviously this is the work of the Skrull "Scott Aaronson", whose thinking is not so clear.

Also, almost every question is so broken as to make answering it completely futile. So much so that it's hard to believe it was an accident.

I find it hard to believe that you could really think the most likely explanation of the flaws you perceive are that Aaronson and the students that implemented this purposely introduced flaws and are trying to sabotage the work. So why do you utter such nonsense?

And did it not occur to you that disagreeing that children should have the vote could be resolved by being neutral on everybody having the vote, which is what I did after realizing that there are plausible interpretations under which I would disagree and plausible interpretations under which I would agree.

Whether you consider this as sabotage or not depends on what you think the goal of the site's authors was. It certainly wasn't to help find inconsistencies in people's thinking, given the obvious effort that went into constructing questions that had multiple conflicting interpretations.

there are plausible interpretations under which I would disagree and plausible interpretations under which I would agree.

Quite.

I got this conflict between my acceptance of the draft in the unlikely event it would be useful, and my belief that all acts I think the Government should be allowed to do are currently allowed. It doesn't seem to know of the existence of this supreme court ruling

Interesting link. I played with it for a while. It kept misunderstanding the nuances of my responses, telling me I was wrong when I wasn't then refusing to listen to my replies. So I stopped playing with it. Two in one day. What are chances?

Good idea, bad implementation. Right now it thinks I have this "tension", but I'm pretty sure it's not a tension.

If the mind is physically independent of the material body but the physical world is closed, empirical observation of the material body cannot be sufficient to determine the existence of a mind.

Versus:

If the program is [abstractly] independent of the [particular] material computer but the physical world is closed, empirical observation of the material computer cannot be sufficient to determine the existence of a running program.

It's the [...] that hurts. "It is possible for one's mind to exist outside of one's material body." does not imply "the mind is physically independent of the material body". It's physically dependent and abstractly independent.

I did have some difficulty resolving all tensions, but I was able to do so. I found that there were often alternate interpretations of a statement that would resolve a tension but were still plausible interpretations. For example, one that I remember was interpreting some of the questions about "physical body" more generally as "physical substrate". Sometimes the tension page didn't offer the question that needed reinterpretation, in which case I deferred the tension until I saw a tension that contained the statement to be reinterpreted.

It definitely does need a lot of work, but I can imagine a tool like this having profound effects on people when all the bugs are worked out and it is applied to mind killers and beliefs/habits where cognitive biases figure prominently.

One major thing that needs to be improved if they intend normal people to use it for normal issues like politics, abortion, etc., is to make the tension page much friendlier. Most LWers have probably studied logic, and can pretty easily interpret the tension explanation, but most people have no clue about logic and won't understand the implicit implications that aren't explained (like that contrapositive of "A -> B" is valid).

Any Santa Fe or Albuquerque lesswrong-ers out there, who might want to chat for an hour? I'll be in Santa Fe for a conference from 9/13 to 9/17, and am flying in and out of Albuquerque, and will have some free time Sunday 9/13.

Just found this note in Shalizi's notebooks which casts an interesting shadow on the Solomonoff prior:

The technical results say that a classification rule is simple if it has a short description, measured in bits. (That is, we are in minimum description length land, or very close to it.) The shorter the description, the tighter the bound on the generalization error. I am happy to agree that this is a reasonable (if language-dependent) way of defining "simplicity" for classifier rules. However, so far as I can tell, this really isn't what makes the proofs work.

What actually makes the proofs go is the fact that there are not very many binary strings of length m for small m. Finding a classifier with a short description means that you have searched over a small space. It's the restriction of the search space which really does all the work. It's not the simplicity of the rules which matters, but the fact that simple rules are scarce in the space of all possible rules. If confines oneself to elaborate, baroque rules, but sticks to a particular style of baroque elaboration, one obtains the same effect.

For context, see the Wikipedia page on PAC learning or this lecture by Scott Aaronson.

Here's my understanding of the dialog, which (as I read it) is not particularly critical of the Solomonoff prior, if that is what cousin_it meant by "casts a shadow".

(Background knowledge) Shalizi understands "Occam's Razor" to be something like "In order to reach the truth, among the theories compatible with the evidence, chose the simplest".

There is a claim that he wishes to refute. The claim is that a certain result is an explanation or proof of Occam's Razor. The result says that if one finds a simple classification rule which works well in-sample, then it is highly probable that it will continue to work well out-of-sample.

This is a failure of relevance. Occam's Razor, as Shalizi understands it, is a way of obtaining TRUTH, but the proof only concludes something about GENERALIZATION PERFORMANCE. To illustrate the difference, he points to an example where, in order to increase generalization performance, one might decrease truth.

Shalizi contrasts the algorithmic information theory proof with Kevin T. Kelly's Ockham Efficiency Theorem, which seems to Shalizi more productive. In particular, Kevin T. Kelly's formalization does talk about truth rather than generalization performance.

Finally, Shalizi provides an alternative ending to the algorithmic information theory proof. If instead of choosing the simplest classification rule, one chose the simplest rule within a sparse random subset of rules (even a non-computable random subset), then you could still conclude a bound on generalization performance. By providing an alternative ending, he has constructed an alternative proof. Presumably this alternative proof does NOT seem like it is a formalization of Occam's Razor. Therefore, the interpretation of the original proof as demonstrating some version of Occam's Razor must also be mistaken.

To sum up, Shalizi is arguing that a certain rhetorical/motivational/interpretational notion which often occurs near a specific proof is wrong. I don't think he's concluding anything at all about the Solomonoff prior.

Looks like I almost missed a very interesting discussion. Hope I'm not too late in joining it.

As far as I can tell, you still need the Solomonoff prior for decision making. Kevin T. Kelly's Ockham Efficiency Theorem says that by using Ockham's Razor you minimize reversals of opinion prior to finding the true theory, but that seems irrelevant when you have to bet on something, especially since even after you've found the truth using Ockham's Razor, you don't know that you've found it.

Also, I think there's a flaw in Shalizi's argument:

Suppose one of these rules correctly classifies all the data.

But if you're working with a sparse random subset of the rules, why would any of them correctly classify all the data? In algorithmic information theory, the set of rules is universal so one of them is guaranteed to fit the data (assuming the input is computable, which may not be a good assumption but that's a separate issue).

All good points if the universe has important aspects that are computable, which seems uncontroversial to me. Thanks for the link, I'd lost it sometime ago, that thread is pretty epic.

If confines oneself to elaborate, baroque rules, but sticks to a particular style of baroque elaboration, one obtains the same effect.

The point of Kolmogorov complexity is that there is some limit to how baroque your rules can ever become, or how baroque you can make a fundamentally simple rule by choosing a really weird prior.

In algorithmic information theory, the problem of choosing a prior is equivalent to choosing a particular universal Turing machine. If you pick a really weird Turing machine, you will end up assigning low prior probability to things that "normal" people would consider simple - like a sine wave, for example. But because of universal computation, there's a limit to how low a probability you can assign. Your weird machine is still universal, so somehow or other it's got to be able to produce a sine wave, after whatever translation-prefix machinations you have to perform to get it to act like a normal programming language.

Another way of viewing this is just to eschew the notion of generalization and state that the goal of learning is compression. If you do this, you end up doing all the same kinds of work, with many fewer philosophical headaches. Now the great deep problem of picking a prior boils down to the rather more quotidian one of picking a data format to use to transmit/encode data.

But because of universal computation, there's a limit to how low a probability you can assign.

I don't understand this claim. Surely we can make the probability of a specific program as low as we want by restricting the programming language in ad hoc ways, while letting it stay Turing complete.

I agree there is some philosophical greyness here. But let me try again.

Let's say we are adversaries. I am going to choose a data set which I claim is simple, and you are going to try to prove me wrong by picking a weird Turing machine which assigns the data a low probability. I generate my data by taking T samples from a sine wave. You pick some strange Turing machine which is designed to be unable to produce sine waves. But regardless of the choice you make, I can always just crank up T to a high enough value so that the compression rate of the data set is arbitrarily close to 100%, proving its simplicity.

cousin_it quoting Shalizi:

It's not the simplicity of the rules which matters, but the fact that simple rules are scarce in the space of all possible rules. If confines oneself to elaborate, baroque rules, but sticks to a particular style of baroque elaboration, one obtains the same effect.

But a particular style of baroque elaboration is one that has a short description.

Not necessarily. (Or did I misread your comment?) The particular style can have an arbitrarily long/complex description, and learning will still work as long as the class of described rules is small enough. This observation seems to imply that algorithmic simplicity doesn't play the central role I'd imagined it to play. This is precisely the point where I'd like to hear LW's informed replies.

Not necessarily. The particular style can have an arbitrarily long description, and learning will still work as long as the class of described rules is small.

Fixed overheads are ignored in the definition of Kolmogorov complexity.

A "particular style of baroque elaboration" is simply a program through whose eyes certain rules look short, which look elaborate and baroque through the eyes of another program.

The PAC-learning scenario doesn't have any parameter that goes to infinity, so I'm not sure why you dismiss "fixed" overheads :-)

The PAC-learning scenario doesn't have any parameter that goes to infinity, so I'm not sure why you dismiss "fixed" overheads :-)

Once you've chosen them, they're fixed, and don't run off to infinity.

This definition-only-up-to-a-constant is one of the weaknesses of minimum description length. (The other is its uncomputability. Shalizi somewhere else remarks that in discussions of algorithmic complexity, it is traditional to solemnly take out Kolmogorov complexity, exhibit its theoretical properties, remark on its uncomputability, and put it away again before turning to practical matters.)

This is precisely the point where I'd like to hear LW's informed replies.

FWIW, this is mine, informed or otherwise. Anyone else have light to shed?

No, let me try nailing this jelly to the wall once again. The definition-only-up-to-a-constant is a weakness of MDL, but this weakness isn't relevant to my question at all! Even if we had some globally unique variant of MDL derived from some nice mathematical idea, learning theory still doesn't use description lengths, and would be perfectly happy with rules that have long descriptions as long as we delineate a small set of those rules. To my mind this casts doubt on the importance of MDL.

Consider this alternative characterization. Someone wants to fit a polynomial to some data. They pre-selected a sparse set of polynomials, which are in general ridiculously complex. Against all odds, they get a good fit to the training data. This theorem says that, because they haven't examined lots and lots of polynomials, they definitely haven't fallen into the trap of overfitting. Therefore, the good fit to the training data can be expected to generalize to the real data.

Shalizi is saying that this story is fine as far as it goes - it's just not Occam's Razor.

Good characterization. It's worth noting that learning theory never gives any kind of guarantee that you will actually find a function that provides a good fit to the training data, it just tells you that if you do, and the function comes from a low-complexity set, it will probably give good generalization.

learning theory still doesn't use description lengths, and would be perfectly happy with rules that have long descriptions as long as we delineate a small set of those rules

Any delineation of a small set of rules leads immediately to a short description length for the rules. You just need to encode the index of the rule in the set, costing log(N) bits for a set of size N.

Note that MDL is not the same as algorithmic information theory (definition-up-to-a-constant comes up in AIT, not MDL), though they're of course related.

You just need to encode the index of the rule in the set, costing log(N) bits for a set of size N.

I see everyone's assuming that some things, by their nature, always go to infinity (e.g. number of samples) while others stay constant (e.g. rule set). This is a nice convention, but not always realistic - and it certainly wasn't mentioned in the original formulation of the problem of learning, where everything is finite. If you really want things to grow, why don't you then allow the set itself to be specified by increasingly convoluted algorithms as N goes to infinity? Like, exponential in N? :-) Learning can still work - in theory, it'll work just as well as a simple rule set - but you'll have a hard time explaining that with MDL.

(If there's some theoretical justification why this kind of outrageous bloatware won't be as successful as simple algorithms, I'd really like to hear it...)

You might find what you're looking for in Kevin T. Kelly's work:

http://www.andrew.cmu.edu/user/kk3n/ockham/Ockham.htm

Dr. Shalizi mentioned it as an alternative formalization of Occam's Razor.

Johnicolas, thanks. This link and your other comments in those threads are very close to what I'm looking for, though this realization took me some time.

To my mind this casts doubt on the importance of MDL.

I think its uncomputability already does that. When you make a computable version by limiting attention to some framework of descriptive capabilities smaller than universal computation, different choices of that framework will give you different measures of simplicity. What is simple in one framework may seem elaborate and baroque in another. Or as some military strategist once put it:

"To the foot-soldier, the strategy of a general may seem obscure, shrouded in shadows and fog, but to the general himself, his way is as plain as if he were marching his army down a broad, straight highway."

I'm going to use this open thread to once again suggest the idea of a Less Wrong video game.

( Here's a link to the post I made last month about it )

After some more thought, I realized that making a game with fancy graphics and complex gameplay would probably not be a good idea for a first project to try.

A better idea would be a simple text-based game you play in your browser, probably running on either PHP or Python.

This might not have as much fun appeal as a traditional video game, since it would probably look like a university exam, but it could still be an effective way for us to measure our rationality.

So far I thought of the following ideas:

1) Multiple Choice questions, with only one right answer. Not just a quiz of trivia you memorized from Less Wrong, but applications of the techniques taught here. Also, the standard tests that reveal if you've learned how to not fall for the standard biases.

2) Questions that require you to do some math, and enter your answer, which can be scored on how close it is to the correct answer.

3) A "spot the logical fallacy" game, or a "spot the bias" game. The player is presented with a few paragraphs of text. It could be a news article, a section of a short story, a conversation between two people, or maybe something else. The purpose is to spot the first sentence that contains a logical fallacy, or a bias. Once you select which sentence contains the error, you have to say what type of error it is. Maybe typing in the name of the fallacy, or maybe selecting it from a list of the fallacies that have pages about them on the LW Wiki.

We could also implement questions that start with an introduction that you can read before the timer starts ticking, then you proceed to the timed part, where you are scored based on how quickly you answer. Maybe some questions could have a time limit of a few seconds, to force an immediate response.

This LW game could be integrated with the rest of the LW website, where your score is linked to your username. Though this feature is entirely optional. The wiki already has a separate user account not linked directly to your main LW user account, so the game could do the same. The game would keep track of which questions you got right, and how long it took you to answer. This would give speedreaders an advantage, but maybe this isn't a problem, because speedreading is a generally useful skill to learn.

The game should keep track of which questions you've already answered, so it doesn't ask you the same question again. Or maybe for the questions where you need to do math, it could ask you again with randomized numbers. Maybe there could be a calculator built into the question webpage, with functions for calculating probabilities, etc.

There should also be a way to compare your scores with other players, or see how you rank.

To generate questions for the game, we could use a comments thread in the main LW site, or we could use the LW wiki, or we could set up a special submission form in the game itself.

another random idea: If you give an answer that you think is right, but the game says is wrong, there should be a way to enter your explanation for why your answer is right. Someone would then check these messages, and if your explanation is valid, the game could award you double points.

(Another random idea I had was to make a text adventure game, where you participate in conversations, and sometimes need to interrupt a conversation to point out a logical fallacy, to prevent the conversation from going off-track and preventing you from getting the information you needed from the conversation. But that would be more of a challenge to implement, and generate content for.)

This could be used to make a game based off of Dungeons and Discourse.

When you attack, you have to select an argument without a flaw, or it gets blocked. When the opponent attacks, if you find a flaw, it deals no damage.

3) A "spot the logical fallacy" game, or a "spot the bias" game. The player is presented with a few paragraphs of text. It could be a news article, a section of a short story, a conversation between two people, or maybe something else. The purpose is to spot the first sentence that contains a logical fallacy, or a bias. Once you select which sentence contains the error, you have to say what type of error it is. Maybe typing in the name of the fallacy, or maybe selecting it from a list of the fallacies that have pages about them on the LW Wiki.

This has already been done many times as part of critical thinking courses; people don't use free sites like http://www.wwnorton.com/college/phil/logic3/ because they're boring and hard.

I think the problem is that we lack a good game mechanic. Come up with a mechanic, and the goals can follow, but it's hard to go from a goal like 'calibrate yourself to avoid overconfidence' to a fun game. We need to think about how to borrow games like Zendo and repurpose them.

Thanks for the link.

That LogicTutor site you linked to provides a good, basic introduction to a few concepts and fallacies. However, the practice problems just ask you to identify which fallacy is in the sentence they give you. They're missing the other half of the game, which is spotting the fallacy in a block of text that's deliberately designed to hide the fallacy. I'll keep looking in case someone has already made a game that contains this part.

One way to make the game more fun would be to have interesting text to find the fallacy in. Eliezer's short stories are a good example of this. Though for the purpose of the game, we would need just short segments of stories, which contain one clear example of a fallacy. Preferably one that's well-hidden, but obvious once you see it. Also, to keep players on their toes, we could include segments that don't actually contain a fallacy, and players would have the option of saying that there is no fallacy.

And as I mentioned before, another idea is to flesh out the stories even more, so that it could be expanded into a mystery game, or an adventure game, or an escape-the-room game, where in order to continue you need to talk to people, and some of these people will give inaccurate information, because they didn't notice a flaw in their own reasoning, and you will need to point out the flaw in their reasoning before they will give you the accurate information.

You would also have to choose your replies during the conversation, and have to choose a reply that doesn't introduce a new fallacy and send the conversation off in the wrong direction. Many of the possible responses would be to question why the person believes specific things that they just said. Maybe there could also be a feature where you could interrupt the person in the middle of what they're saying, to point out the problem. Optionally, score the player based on how long they took, and how many wrong paths they went down before finding the correct path.

And if this still doesn't make the game fun, then there are ways to make people want to play games even if they aren't fun.

One way is to make the not-so-fun game into a small but necessary part of a bigger game, that is fun.

Another way is to make it into a Flash game, and submit it to a site that tracks your achievements in the games, and gives you an overall score over all the games you've played. Examples of sites like this are Kongregate.com, and Newgrounds.com. If you're lucky, the game might even get to spend a few days on the site's front page, or get a limited-time challenge, for extra bonus points.

If the game could be made into a series, that would be even better.

So far I've only seen one "educational" game get promoted to the front page of Kongragate.com, with its own limited-time-only challenge. That was Globetrotter XL. The game says the name of a city, and you have to click on where that city is, on a map of the world with no border lines or labels. You're scored for how close you click to the actual city. Unfortunately, I ended up not completing this game's limited-time challenge, because the game was so unforgiving, and my knowledge of geography was so poor.

Anyway, while it would be a really nice bonus if the spot-the-fallacy game became popular outside LW, the main purpose of the game is to give current LW members an objective way to test their rationality, even if the game isn't especially fun.

There have been several LW posts now talking about how desperately we need a way to measure our rationality, and so far I haven't seen any serious proposals that anyone is actually working on. Or maybe there's a project already started, and I just didn't notice it because I got so far behind on reading the LW posts.

This game/quiz/whatever is a proposal for a way to go from having no objective way to measure our rationality, to at least having something.

And as I mentioned before, another idea is to flesh out the stories even more, so that it could be expanded into a mystery game, or an adventure game, or an escape-the-room game, where in order to continue you need to talk to people, and some of these people will give inaccurate information, because they didn't notice a flaw in their own reasoning, and you will need to point out the flaw in their reasoning before they will give you the accurate information.

Many of the possible responses would be to question why the person believes specific things that they just said.

Well that would be an interesting game mechanic.

Incidentally, another link right up your alley: http://projects.csail.mit.edu/worldview/about

(Starting to think maybe we could use a wiki page, even if only for links and ideas. This game discussion is now spread out over something like 5 LW articles...)

5 articles? I only know of 2 articles where this topic was discussed. Can you post a link to the other 3, please?

Anyway, I went ahead and created a wiki page for discussing the game idea, and posted links to the two threads I know of:

http://wiki.lesswrong.com/wiki/The_Less_Wrong_Video_Game

It certainly feels like it's been spread out over more than 2 open threads!

Out of pure curiosity, what's your probability distribution of Scientology (or some other such group) being useful? Not the Xenu part, but is it possible that they've discovered some techniques to make people happier, more successful, etc.?

We already have some limited evidence that conventionally religious people are happier, and conventional religions are quite weak.

We already have some limited evidence that conventionally religious people are happier

But see Will Wilkinson on this too (arguing that this only really holds in the US, and speculating that it's really about "a good individual fit with prevailing cultural values" rather than religion per se).

That's a good counter-argument, but the linked post doesn't actually measure religion-happiness correlation within those other countries (which is the relevant factor), and it's very plausible that European monopolistic religions are far less effective than American freely competing religions for creating happiness.

The Snoep paper Will linked to measured the correlation for the US, Denmark and the Netherlands (and found no significant correlation in the latter two).

The monopolist religion point is of course a good one. It would be interesting to see what the correlation looked like in relatively secular, yet non-monopolistic countries. (Not really sure what countries would qualify though.)

I'm going to completely ignore "statistical significance", as scientific papers are well known to have no idea how to do statistics properly with multiple hypotheses, and can be assumed to be doing it wrong until proven otherwise.

If null hypothesis were false, the chance of all almost signs pointing in the same direction would be very low. As far as I can tell what the paper finds out is that religion is less effective in Denmark and Netherlands than in US, but it increases happiness, and it's extremely unlikely to be a false positive result due to chance.

I know a scientist who's spent significant money on Landmark Education (nee "est"). He's happy with what he got out of it, but doesn't feel the need for any more of it now.

I'm curious about how Less Wrong readers would answer these questions:

  1. What is your probability estimate for some form of the simulation hypothesis being true?

  2. If you received evidence that changed your estimate to be much higher (or lower), what would you do differently in your life?

To answer both, there's no consequence. So I choose not to invent a completely arbitrary prior.

I do enjoy fantasizing about possible measurable consequences of particular types of simulations. Perhaps if I'm interesting enough, I'll be copied into other simulations; perhaps we can discover some artifact of variably approximate simulation when no important observer is near, etc.

A simulation hypothesis such as "our universe is a simulation" is not falsifiable even given perfect knowledge of the universe at some point in time; maybe the universe has a definite beginning and end and it's simulated perfectly the whole way through. Therefore, I'll use the following definition of the simulation hypothesis: "The best description of the universe as we are capable of observing it describes our observations as happening entirely within a simulation crafted by optimizing processes."

Let's assume for the sake of convenience that "the" priors for the laws of physics are P, and let's call the distribution of universes that optimizing processes would simulate P'. The only necessary difference between P and P' is that P' is biased toward universes that are easy and/or useful to simulate. How easy a universe is to simulate in general can probably be estimated by how easy a universe is to simulate in itself. We have quantum mechanics but quantum computers have been late in coming, suggesting that our universe would be difficult to simulate. Now, as for utility, evolution optimizes for things that themselves optimize for reproduction, but it also produces optimization for pretty much random things. We can ignore the random things, and ask how useful our universe is for reproduction. I'm guessing that the universe, as it seems to involve lots of pointless computation, is not good for that.

So, given the above, I'd estimate the probability as being... oh, how does 20% sound?

Now, of course, the other thing to look for in a simulated universe is simulation artifacts: things that seem to not follow the laws of physics, and behaviors that are only approximations to how things should behave. Suffice to say, we haven't seen any of those.