All of Brickman's Comments + Replies

Harry Potter and the Methods of Rationality discussion thread, part 16, chapter 85

I thought of a possible reason why they wouldn't do this. Basically, you've got two choices with the unbreakable vow ploy: Obtain a class of civil servants willing to give up their own magic to do the job (fat chance), or force the criminals to do it to each other. The natural answer is the latter right? Well yes, except the part where you have to hand a criminal whose crime was severe enough to warrant stripping some of his magic a wand and give him enough mental breathing room to perform a complicated, powerful ritual. Some of them are just gonna go along with it, sure, but you only gotta have one high-profile screwup before that kind of a policy is abolished.

5wedrifid9yOr, you could think strategically for a few minutes then, for example, only give magic wands to wizards for this purpose after they have themselves sworn unbreakable vows that would prevent any misuse.
Harry Potter and the Methods of Rationality discussion thread, part 16, chapter 85

Whatever theories we may have, most of them contingent on the defense professor being Voldemort and the one behind this plot (a conclusion Harry hasn't yet reached and which, frankly, there isn't enough in-universe or possibly even total evidence to make conclusive), it is NOT "obvious" that noone was meant to die. Draco almost died and if it was anyone except the defense professor behind this he was almost certainly supposed to die. Hermione was, as far as I can tell, being sentenced to life in Azkaban, which I'd rank about the same as killing her. If anything, it's obvious that this was meant to be a lethal plot and sets him pretty firmly on the "defect" side.

2drethelin9y10 years in azkaban for attempted murder.
6pedanterrific9yShe was being sentenced to ten years in Azkaban, which is the same thing as "death by slow torture".
Harry Potter and the Methods of Rationality discussion thread, part 16, chapter 85

Morally he didn't do it, and maybe Quirrel even had a desire to kill her sitting on a back burner before Harry got involved, but her death was caused by her interaction with Harry. It is no stretch to say that there is at least one hypothetical sequence of actions Harry could have taken, even given knowledge at the time (not realizing she worked for Lucius or was an animagus) which would not have resulted in her death. Heck, doing nothing would have resulted in her not death.

That is the level of challenge Harry is taking upon himself. Not just to not kill... (read more)

but a single nameless innocent bystander who catches a Cutting Curse

It seems that he promised himself to stop trying to save everyone even if a minor character dies accidentally. In that case it wouldn't matter if he considered himself directly responsible for the death of Rita Skeeter.

You can't do that.

Indeed. I don't see how he could manage not to compromise his 'every human life is precious' principle in a war. He's hesitating between two possible courses of action -- doing the math or playing Ghandi -- and neither seems like a satisfying choice. He really needs to become omnipotent or at least avoid the necessity of making such a choice.

0[anonymous]9yIt seems that he promised himself to stop trying to save everyone even if a minor character dies accidentally. In that case it wouldn't matter if he considered himself directly responsible for the death of Rita Skeeter. Indeed. I don't see how he could manage not to compromise his 'every human life is precious' principle in a war. He's hesitating between two possible courses of action -- doing the math or playing Ghandi -- and neither seems like a satisfying choice. He really needs to become omnipotent or at least avoid the necessity of making such a choice. Edit: Oops, I messed up quotes and accidentally retracted the comment
Harry Potter and the Methods of Rationality discussion thread, part 16, chapter 85

Is it me, or does Harry's solution to this dilemma seem rather... half-assed? Ignoring potential the loss of effectiveness from his resolving to suddenly switch directions the first time things get bad, is he really going to know the first time someone dies as a result of his war? How will he know the difference? He's already gotten someone killed by his actions (Rita Skeeter, who he doesn't even know about) and another person gravely injured (that auror hurt by the rocket, who he doesn't know about but admittedly he thought the whole affair was a mistake ... (read more)

2Swimmy9yYour opportunity cost point is more obvious to me than your Rita Skeeter point. Harry just sacrificed several lives, not just in people he could save today but almost certainly in people he could have saved once the war started. Potentially justified if Hermione is nigh-irreplaceable in the project of discovering the underlying structure of magic, which might give a hint as to where the plot's going. But I'm not sure Harry could reasonably predict that.

He's already gotten someone killed by his actions (Rita Skeeter, who he doesn't even know about)

Not for any realistic sense of the phrase 'by his actions'. Quirrel squished Rita of his own accord for his own purposes and Harry's presence there is damn near irrelevant.

Harry Potter and the Methods of Rationality discussion thread, part 15, chapter 84

Personally, to me it struck me as something I would try on a group of first graders (provided I knew I wouldn't be sued) but not on a group of adults. They know it's just a game, and treat it as such, but nobody's going to refuse because to me that sounds like a very fun game (approached from the right mindset, anyways, and provided you make sure the audience doesn't take it too far. I'd probably hand pick people to "criticize" and make myself a member of that group so I could step in if another was being problematic). So they all do it, and they... (read more)

4pedanterrific9yThere's almost certainly a way to do this sort of thing that would function quite well as a morale and team-building exercise even if it didn't work at all on the conformity level. Not that the Chaos Legion needs more morale.
Harry Potter and the Methods of Rationality discussion thread, part 14, chapter 82

"Ah!" Harry said suddenly. "I get it now. The first False Memory Charm was cast on Hermione after Professor Snape yelled at her, and showed, say, Draco and Professor Snape plotting to kill her. Then last night that False Memory was removed by Obliviation, leaving behind the memories of her obsessing about Draco for no apparent reason, at the same time she and Draco were given false memories of the duel."

Since that was the last theory Harry proposed before he switched from theories to lines of attack, and nobody fully shot it down (there... (read more)

Harry Potter and the Methods of Rationality discussion thread, part 14, chapter 82

Personally, for anything except comedy I like to read moderately-long bursts rather than short snippets--when I follow works that update daily but have updates that are too small I often stop reading for a while on the assumption that when I start again later I'll have a juicy backlog to trawl through. (Part of me wonders if it's just a matter of how good the author is at finding good stopping points though). HPMOR updates are not that small, but with its plot-heaviness I think I still found that I enjoyed it better when I read entire arcs at once than whe... (read more)

Harry Potter and the Methods of Rationality discussion thread, part 14, chapter 82

Personally my problem with Harry wasn't so much that he immediately assumed there was a trick (shouldn't get a probability of 1.0, no, but certainly a basket worth piling some eggs in) but that he assumed the truth would get her off. He never once stopped and asked Dumbledore and Snape "If it was proven that she had been tricked into doing this with false memories, but still cast the spell willingly and with her own hand, would the Wizengamot still convict her?" I don't even know the answer to that question, but I'd certainly ask before I assumed... (read more)

5Eugine_Nier9yHarry doesn't know about the GHD attack and so his working hypothesis is that her memory of attempting to kill Draco is false.
Harry Potter and the Methods of Rationality discussion thread, part 14, chapter 82

I don't think that "Lucius chose the exact same number as a stab against Dumbledore" is a very complex hypothesis. We already know that he knows part of the story and can reasonably assume he knows the whole story about Aberforth. So of course if the situation already demands that he hold someone on Dumbledore's side (sort of) for ransom for some obscene amount of money, on the assumption that it won't be paid, how could he resist rubbing that bit of salt in his nemesis's wounds?

It's not part of some bigger plan. It's not some fancy maneuver. It's just an emotional attack of opportunity aimed at Dumbledore, probably just for pride's sake.

Harry Potter and the Methods of Rationality discussion thread, part 13, chapter 81

My interpretation has been more that the 'dark' plans rely primarily on application of force (most often political rather than physical)--threatening, blackmailing, bribing--and trickery. They tend to work in the short run, but in the long run can poison his reputation (people notice how dark he acts over time) and have nasty side effects. For the most part Harry's dark plans are pretty clever, because his dark side is pretty ruthless and very clever.

If you take that definition for the plans his dark side comes up with, he actually started out with a light... (read more)

Harry Potter and the Methods of Rationality discussion thread, part 13, chapter 81

I don't think Harry actually would have taken Dumbledore as an enemy if Dumbledore failed to save Hermione, as he clearly was trying and even using up political capitol. Only having Dumbledore stand in the way of Harry saving her would do that, and when Dumbledore realized just how determined Harry was he had the sense to step aside.

Also I'm not really sure how well "Delegitimized the Wizengamot in the eyes of Magical Britain" would have worked--rest of the world yes, but the papers were certainly doing a hatchet job on her. The question is how ... (read more)

0see9yCertainly the level of wedge between Harry and Dumbledore if Dumbledore let something really bad happen to Hermione, versus the amount of political capital Dumbledore would have had to spend/lose helping Hermione, covered a broad range of probabilities. I put it in maximal terms, mostly to put it in sharp relief; the precise costs would have been uncertain, but almost certainly real to some extent. Some advantage gained in any case, if not devastating. And I will freely grant that the percentage of "Magical Britain horrified at the sentence" is hard to determine, but I think 5% is a solid lower bound; some advantage if not big advantage. Similarly, , per Spurlock's comment, wiping out Hermione as "a Light-side witch showing skill at military command and Battle Magic", well, it was obviously not enough of a consideration to keep him from making Hermione a general in the first place . . . but even then it's still at least some advantage. And so on. There are, I think, reasonable arguments for minimizing many of these items, but even then you still wind up with a long list of small advantages, and a lot of small advantages in itself adds up to a big advantage. What's also interesting is how many of these got at least partly achieved even though Hermione basically is going unpunished. QuirrelCloakMort's plan may not have achieved everything it could have, but it didn't fail.
Measures, Risk, Death, and War

I'm not sure it's appropriate to consider the money the average human will accept for a micromort as a value that's actually useful for making rational decisions, because that's a value that's badly skewed by irrational biases. Actions are mentally categorized into those the thinker does and doesn't believe (on a subconscious level) to possibly lead to death. I doubt the average person even considers a "risk" factor at all when driving their car or walking several blocks to the car (just a time factor and a gasoline factor), unless their trip tak... (read more)

1Vaniver9yRight. This is why coming up with your own value is a good thing to do. (I didn't talk much about it in the post because it's highly personalized; I didn't want to work through it for all sensible utility functions, and describe how to pick the parameters, and because I didn't want to do it for all of them I didn't want to describe it for just one, because that wouldn't be appropriate for a majority of readers, I suspect.) Yep, which can cause people to behave suboptimally. One of the main values of this sort of analysis is it gives you a "risk cost" to put together with a "time cost" and a "gasoline cost." The weekly game night that I drive to costs me $1.40 in risk, $3.20 in gas, and about $6 in time- so the risk is actually a pretty small factor there, but it could tip the scales for marginal activity. (You do need to look up the mortality numbers- which can have a non-trivial cost- but doing research when it's worth it is a part of careful decision making.) I'm not sure how the EPA runs their numbers, but the way I got mine was by calculating the value of my life (on the margin). I think people can give reasonable answers for things like "how much longer would your life have to be to compensate you for a 5% decrease in consumption?", which is less subject to biases than visualizing particular causes of death.
Funnel plots: the study that didn't bark, or, visualizing regression to the null

Sadly, your commitment to this goal is not enough, unless you also have a guarantee that someone will publish your results even if they are statistically insignificant (and thus tell us absolutely nothing). I admit I've never tried to publish something, but I doubt that many journals would actually do that. If they did the result would be a journal rendered almost unreadable by the large percentage of studies it describes with no significant outcome, and would remain unread.

If your study doesn't prove either hypothesis, or possibly even if it proves the nu... (read more)

0dlthomas9yThis depends on how it was organized. Data sets could be maintained, and only checked when papers show interesting results in nearby areas.
3Dan_Moore9yIf I can't get this study published in the traditional way, I'll "publish" it myself on the internet. In this case, what I'm calling the null hypothesis is somewhat meatier than a null hypothesis you would typically find in a medical study. The voluntary supplemental financial reporting for these (insurance) companies (starting with 2011) is something called market consistent embedded value (MCEV). My null hypothesis is that the phrase 'market consistent' is accurate - this is roughly equivalent to assuming that, in valuing the long-term liabilities of these companies, market participants pretend that they are securities with the same cashflows. My alternate hypothesis is that market participants value these liabilities within a framework of the company as a going concern, focusing on the company's cost of meeting these liabilities.
6TimS9yYou might find this discussion of the publication of a null result paper interesting. [http://chrisblattman.com/2011/11/28/the-publication-bias-problem-and-the-redemption-of-blattman/]
A few analogies to illustrate key rationality points

I like the first two, and the chess one's pretty interesting though I can't imagine I'd have an easy time getting someone to stand still long enough to hear the whole thing as an argument. But I don't really like the last one. You've been tricked into accepting his premise, that death lets you create more meaningful art, and trying to regain ground from there. It's that premise itself that you should be arguing against--point out all the great literature and art that isn't about death, and that you could still have all of that once death was gone. Also poi... (read more)

Also point out that to someone with cancer today the availability of art is probably less valuable than the availability of a cure would be.

Or to approach the same point from a slightly different direction - Elie Wiesel wrote some pretty awesome stuff, but that doesn't mean we should have more Holocausts.

The Optimizer's Curse and How to Beat It

Oh, I understand now. Even if we don't know how it's distributed, if it's the top among 9 choices with the same variance that puts it in the 80th percentile for specialness, and signal and noise contribute to that equally. So it's likely to be in the 80th percentile of noise.

It might have been clearer if you'd instead made the boxes actually contain coins normally distributed about 40 with variance 15 and B=30, and made an alternative of 50/1, since you'd have been holding yourself to more proper unbiased generation of the numbers and still, in all likelih... (read more)

0CynicalOptimist4yI think there's some value in that observation that "the all 45 thing makes it feel like a trick". I believe that's a big part of why this feels like a paradox. If you have a box with the numbers "60" and "20" as described above, then I can see two main ways that you could interpret the numbers: A: The number of coins in this box was drawn from a probability distribution with a mean of 60, and a range of 20. B: The number of coins in this box was drawn from an unknown probability distribution. Our best estimate of the number of coins in this box is 60, based on certain information that we have available. We are certain that the actual value is within 20 gold coins of this. With regards to understanding the example, and understanding how to apply the kind of Bayesian reasoning that the article recommends, it's important to understand that the example was based on B. And in real life, B describes situations that we're far more likely to encounter. With regards to understanding human psychology, human biases, and why this feels like a paradox, it's important to understand that we instinctively tend towards "A". I don't know if all humans would tend to think in terms of A rather than B, but I suspect the bias applies widely amongst people who've studied any kind of formal probability. "A" is much closer to the kind of questions that would be set as exercises in a probability class.
0Manfred10yThat's true - when I wrote the post you replied to I still didn't really understand the solution - though it did make a good example for JGWeissman's question. By the time I wrote the post I linked to, I had figured it out and didn't have to cheat.
The Optimizer's Curse and How to Beat It

I'm trying to figure out why, from the rules you gave at the start, we can assume that box 60 has more noise than the other boxes with variance of 20. You didn't, at the outset of the problem, say anything about what the values in the boxes actually were. I would not, taking this experiment, have been surprised to see a box labeled "200", with a variance of 20, because the rules didn't say anything about values being close to 50, just close to A. Well, I would've been surprised with you as a test-giver, but it wouldn't have violated what I unders... (read more)

2Manfred10yThe key factor is that the 60,20 box is not in isolation - it is the top box, and so not only do you expect it to have more "signal" (gold) than average, you also expect it to have more noise than average. You can think of the numbers on the boxes as drawn from a probability distribution. If there was 0 noise, this probability distribution would just be how the gold in the boxes was distributed. But if you add noise, it's like adding two probability distributions together. If you're not familiar with what happens, go look it up on wikipedia, but the upshot is that the combined distribution is more spread out than the original. This combined distribution isn't just noise or just signal, it's the probability of having some number be written on the outside of the box. And so if something is the top, very highest box, where should it be located on the combined distribution? Now, if you have something that's high on the combined distribution, how much of that is due to signal, and how much of it is due to noise? This is a tougher question, but the essential insight is that the noise shouldn't be more improbable than the signal, or vice versa - that is, they should both be about the same number of standard deviations from their means. This means that if the standard deviation of the noise is bigger, then the probable contribution of the noise is greater. Me saying the same thing a different way can be found here [http://lesswrong.com/lw/76k/the_optimizers_curse_and_how_to_beat_it/4unu].
Harry Potter and the Methods of Rationality discussion thread, part 9

The problem is, Dumbledore's not going to tell Harry what the condition is for getting the stone. Why would he? He didn't tell canon Quirrell, who was standing there trying to figure out why he couldn't get it. He didn't even tell canon Harry until after the fact. The mirror as a screening process works even better if the person being screened doesn't know what it's testing for, and thus can't fake it.

And Harry would want to use the stone, make no mistake. The first thing he'd do with it is make himself immortal, to make sure no accident or fluke could sto... (read more)

0thomblake9yGiven the author, Harry could also be told after getting the stone that he could only get it if he would not use it, and therefore he would not use it. And he's already been cautioned against magical paradoxes.
-1JoshuaZ10yOk. Valid point. But after trying at a few minutes Harry might be able to understand Dumbledore enough to realize what the trick is. On the other hand, that might take far more empathy for the viewpoints of others than Harry generally has.
Harry Potter and the Methods of Rationality discussion thread, part 9

I think you hit on a key point that several are missing--Dumbledore wouldn't want HJEPV to have the stone any more than Quirrell (well, maybe a little more, but certainly less than nobody having it or even than handing it off to, say, some random Hufflepuff). In canon Harry didn't just not want to use it, he didn't want it used--that was his entire motivation for getting it. Rational Harry would, probably quite literally given enough time to think on the situation, kill to use it, and use it repeatedly. And Dumbledore knows this.

Canon Harry was, in fact, a... (read more)

1JoshuaZ10yLoophole: Harry doesn't want to use the stone, he wants to reverse engineer it, and mass produce more. So he can easily commit to not using the stone.
That Tiny Note of Discord

Despite having seen you say it in the past, it wasn't until reading this article that in sunk in for me just how little danger we were actually in of Eliezer1997 (or even Eliezer2000) actually making his AI. He had such a poor understanding of the problem, I don't see how he could've gotten there from here without having to answer the question of "Ok, now what do I tell the AI to do?" The danger was in us almost never getting Eliezer2008, or in Eliezer2000 wasting a whole bunch of future-minded peoples' money getting to the point where he realize... (read more)

Newcomb's Problem and Regret of Rationality

I'm kind of surprised at how complicated everyone is making this, because to me the Bayesian answer jumped out as soon as I finished reading your definition of the problem, even before the first "argument" between one and two boxers. And it's about five sentences long:

Don't choose an amount of money. Choose an expected amount of money--the dollar value multiplied by its probability. One-box gets you >(1,000,000*.99). Two-box gets you <(1,000*1+1,000,000*.01). One-box has superior expected returns. Probability theory doesn't usually encounte... (read more)

"Science" as Curiosity-Stopper

I think you may be placing too much emphasis on curiosity as a terminal value here rather than a means of acquiring other terminal values--not that I think it has no value in and of itself, but that's not its only use and not its biggest in most respects.

If I know that a light switch/bulb's properties are fully explained by science and nothing else about it, that DOES tell me things I didn't know beforehand. It tells me that it is much less of a priority to figure out how the light bulb works than it would have been if nobody had a clue. If there is any si... (read more)