All of botogol's Comments + Replies

if management are doing that then are neglecting a powerful tool in their tool-kit, because announcing a G will surely cause G to fall, and experience says that to begin with a well-chosen G and G remain correlated (because many of the things to do to reduce G also reduce G). It is only over time that G* and G detach.

At work a large part of my job involves choosing G , and I can report that Goodhart's Law is very powerful and readily observable.
Further : rational players in the workspace know full-well that management desire G, and the G
is not well-correlated with G, but nonethelss if they are rewarded on G*, then that's what they will focus on.

The best solution - in my experience - is mentioned in the post: the balanced scorecard. Define several measures G1 G2 G3 and G4 that are normally correlated with G. The correlation is then more persistent : if all four me... (read more)

Why obviously? Are you so afraid that people would do the right thing without immediate incentives? I think I'd measure G1 first, but would tell in advance that next quarter we will measure that one of G1,G2,G3,... which will be most critical at the beginning of that quarter.

That's true (that they have biases) although I understand the training is attend to the nature of the injury, and practicalities of the situation - eg danger to the firefighter - rather than the age of the victim.

However what one might expect to see in firefighters would be ethical dilemmas like the trolley problem to trigger the cerebral cortex more, and the amaglydia less than in other people.


Unless of course the training works by manipulating the emotional response. So firefighters are just as emotional, but their emotions have been changed by their training.

This is the sort of problem Kahane was talking about when he said it is very difficult to interpret brain scans.

It worries me that we do not have more emphasis placed upon mustering out our Armed Forces members to undo some of the training that they receive, simply because their emotional biases have been so changed that it makes it difficult for many of them to re-integrate into society. I think that we are developing a similar problem with Police, who are used to interacting primarily with the worst parts of society, and then developing a bias that the rest of society has similar behavioral trends as that lowest common denominator they are used to seeing. I will have to re-read the Kahane comments about interpreting brain scans...

A person in the audience suggested taking firefighters, who sometimes face dilemmas very like this (Do I try to save life-threatened person A or seriosly injured Baby B), and hooking them up to scans and seeing if their brains work differently - The hypothesis being that they would make decision in dilemmas more 'rationally' and less 'emotionally', as a result of their experience and training. Or the pre-disposition that led to them becoming fire-fighters in the first place.

Of course, just like with Military Training, the Firefighters may have biases about what they consider to be rational. For instance, most would probably save the injured baby at the expense of an uninjured adult or child. Yet, the baby has less immediate worth than the adult or small child, as these latter two are conscious and self-aware in a way that the baby is not. Yet, almost instinctively, humans tend to go for the baby. Of course, genetics has wired us to be that way.

The opening was deliberate - it's a common way that newspaper Diarists start their entries.... but perhaps it's a common way that British newspaper diarists start their entries, and sounds wrong to american ears. So I have changed it. Nations divided by a common language etc.

Yes. People get bogged down with the practical difficulties. Another common one is whether you have the strength to throw the stranger off the bridge (might he resist your assault and and even throw you off).

I think the problem is the phrasing of the question. People ask 'would you push the fat man', but they should ask 'SHOULD you push the fat man'. A thought experiemnt is like an opinion poll, the phrasing of the question has a large impact on the answers given. Another reason to be suspicious of them.

No, I wasn't declaring it meaningless.

My (perhaps trivial) points were that all hypothetical thought experiments are necessarily conducted in Far mode, even when thought experiment is about simulating Near modes of thinking. Does that undermine it a little?


  • while all Thought Experiments are Far
  • Actual Experiements are Near.

I was illustrating that with what I hoped was an amusing anecdote -- the bizarre experience I had last week of having the trolley problem discussed with the fat man actually personified and present in the room, sitting next ... (read more)

:-( no, not a draft! It was just supposed to be light-hearted - fun even - and to make a small point along the way.... it's shame if lesswrong article must be earnest and deep.

I think the thing that made it seem like a draft is the missing "I went" at the beginning of the article. I also noticed illustrate is misspelled, at a quick glance.

no, not at all, I don't think rational = unemotional (and I liked EY's article explaining how it is perfectly rational to feel sad ... when something sad happens).

But rationality does seem to be stongly associated with a constant meta-analytical process: always thinking about a decision, then thinking about the way we were thinking about the decision, and then thinking about the self-imposed axioms we have used to model the way that we were thinking about the meta-thinking, and some angst about whether there are undetected biases in the way that .. yada ... (read more)

I recently started to notice this very question popping up in my head when I find myself in a situation like the one you described :) I didn't consciously install this habit, it just started to manifest itself some time ago, probably several months. But I do find myself in similar situations, usually after a good workday -- just substitute a cup of tea (or a small shot of Becherovka) for the beer, and mindless internet surfing for the wrestlers :) (Related: Ego Depletion: Is the Active Self a Limited Resource? [])

Do you act all rational at home . . or do you switch out of work mode and stuff pizza and beer in front of the TV like any normal akrasic person? (and if you do act all rational, what do your partner/family/housemates make of it? do any of them ever give you a slap upside the head?)


How does it look like when a person "acts rationally"? Do I hear connotations with dreaded Mr. Spock []?
*coughs []*

Can you make a living out of this rationality / SI / FAI stuff . . . or do you have to be independently wealthy?

If you are good enough at the rationality stuff you can make a living in business with it, but you have to be VERY good.
I strongly think that's the wrong way to phrase the question. "Don't expect fame or fortune. The Singularity Institute is not your employer, and we are not paying you to accomplish our work. The so-called "Singularity Institute" is a group of humans who got together to accomplish work they deemed important to the human species, and some of them went off to do fundraising so the other ones could get paid enough to live on. Don't even dream of being paid what you're worth, if you're worth enough to solve this class of problem. As for fame, we are trying to do something that is daring far beyond the level of daring that is just exactly daring enough to be seen academically as sexy and transgressive and courageous, so working here may even count against you on your resume. But that's not important, because this is a lifetime commitment. Let me repeat that again: Once you're in, really in, you stay. I can't afford to start over training a new Research Fellow. We can't afford to have you leave in the middle of The Project. It's Singularity or bust. If you look like a good candidate, we'll probably bring you in for a trial month, or something like that, to see if we can work well together. But please do consider that, once you've been in for long enough, I'll be damned hurt – and far more importantly, The Project will be hurt – if you leave. This is a very difficult thing that we of the Singularity Institute are attempting – some of us have been working on it since long before there was enough money to pay us, and some of us still aren't getting paid. The motivation to do this thing, to accomplish this impossible feat, has to come from within you; and be glad that someone is paying you enough to live on while you do it. It can't be the job that you took to make the rent. That's not how the research branch of the Singularity Institute works. It's not who we are." - [

I have been in Ashley's situation - roped in to play a similar parlour game to demonstrate game theory in action.

In my case it was in a work setting: part of a two day brainstorming / team building boondongle.

In my game there were five tables each with eight people, all playing the same, iterarted game.

In four out of five table every single person cooperated in every single iteration - including the first and last one. On the fifth table they got confused about the rules.

The reason for the behaviour was clear - the purpose of the game was to demonstrate ... (read more)

Pretending to not understand the game and acting embarrassed in order to defect without social consequences seems like a pretty good strategy to me.

"Do not be too moral. You may cheat yourself out of much life. Aim above morality. Be not simply good; be good for something" Thoreau

What a lot of comments (and I was worried that it was all too trivial. Lesson: never underestimate the power of Dr Who) Thanks all.

@Nanani - yes, indeed, the initial round up of 600 or so was composed of waifs and strays like that, inc the ill. But when the demand of 10% was acceded to there wasn't time to handpick

@SharedPhoenix - I agree and a strength of this story was that was no easy way out. The scenario was played out right to the end with the main character forced to make a rational sacrifice. OK, he found a way for it to be jsut one child, but ther... (read more)

Threaded comments are your friend. Just click "reply" on the comment you want to reply to. Bonus: the relevant person will be notified of your response.

perhaps an arm-wrestling contest would be acceptable... hmm, but not possible on bloggingheadstv... a face-pulling contest?

what i'd actually like to see would be Robin Hanson v Mencius Moldbug

And it is done! [] []
I would like to see Mencius Moldbug versus... ...Mencius Moldbug. Not for insight or informational content, but perhaps as a sort of Théâtre de l'Absurde. I think Robin has been right in not wasting his time further.
Robin made it clear that he will not continue a debate with someone who does not show proper deference to white papers.

William Gibson?

He also thinks a lot - and cleverly - about the future but in a different way from Eliezer.

good luck. I am out of town today, but perhaps the next one...

1Paul Crowley14y
I can't spare the time today - have fun!

It all goes to show that

what's grist to the mill is nose to the grindstone.

Browne's description of his own symptoms reminds me of interviews I have read and seen of Terry Pratchett talking about his early-onset dementia - particuarly this

It's unusual because people deal with me and they refuse to believe I have Alzheimer's because at the moment I can speak very coherently, I can plot a novel


Yvain, thanks for this - a fascinating case I hadn't read about before.

I think OB has improved since LW started up. OB now feels calmer and it' better paced.

"I'm pretty sure most of my Christian friends don't believe that any of Genesis is literally true"

Have you asked them? Probably not, it's considered rude to ask christians questions like that, isn't it? (which is no doubt one reason why religious beliefs are able persist)

But if you did ask them you might be surprised by the answer.

Actually I suspect you are probably somewhat right: they don't beleive genesis literally. However I suspect they don't disbelieve it, either.

I actually don't think religious belief has much to to with doctrine, a... (read more)

I also tried manually upvoting my own post - just t see what would happen.

Never never do that.

======== More seriously - shouldn't you get Karma from people REPLYING to your comments? Lots of Karma - I mean: someone upvotes me - that's nice - but someone actually REPLIES to me - wo hoo!

Or, do it twice.
Your own comments have an automatic upvote applied to them - this works like any other upvote. If you click "Vote up" after you've already voted something up, it cancels the upvote. You can (potentially) determine whether you've currently voted a comment up or down based on whether "Vote up" or "Vote down" is in boldface. Trolls and flamebait get lots of replies too, and that shouldn't count towards karma.

OK, so as an ardent game player and natural pedant, I need the rules and scoring sysem of this 'karma' thing explained to me - can I find it on the site somewhere?

To start with: I seem to get a karma point just for making a comment.. is that right?

(or is my mum on-line here upvoting my every post)

1Eliezer Yudkowsky14y
You currently get a karma point just for automatically upvoting your own comment. This is scheduled to be fixed, but it's low-priority.
It's the vote total of all your comments & posts. Lots of info and opinion on it is here []. No, you are. It upvotes on your behalf.

I like this article (but then I liked Dennet's ideas of belief in belief right from the start) and I've been thinking about this off and on all day.

But I think perhaps Eliezer over-analyses: On the surface this person's beliefs and thoughts seem fuzzy, so Eliezer admiraly digs deeper - but perhaps it's just fuzz all the way down.

Perhaps she believes P and ~P, perhaps she believes P>Q and she believes P but she beleives ~Q.

Perhaps you just have to shrug, and move on.

My experience is that most religious people give very, very, very little thoug... (read more)

But for an academic actually doing that analysis (not that I'm necessarily calling EY an 'academic'), one must invoke the principle of charity [], which necessitates assuming she's saying things that are reasonable, justified, and truthful, as far as you can push it. Argue against the belief, not the person - if you can wrestle out some truth from what someone's saying, count that as a win even if they oppose you.
3Paul Crowley14y
What you say doesn't account for the curious absence of any direct affirmation of her belief - it's weird that she's always at one remove from her own belief.

Eliezer asks "how did you come to rationality?" It surprises me how many people answer: "this is how I lost my religion"

Clearly you can't be rationalist, while also being religious, but there is a more to rationality than simply absence of religion..

Anyway... personally: there's no one moment, but I'm a natural born sceptic and persistently urious analyst. Perhaps rationality attracted because it seems like methodical, organised, analytical scepticism

Single biggest book: Hofstadter's G-E-B, right when it first came out. I just didn't know there could be a book like that....