As an aside; the use of "Org" (i.e. Rationality Org) seems really unusual and immediately makes me think of Scientology (Sea Org); am I unusual in having this reaction?
Yikes, thanks for mentioning this; I will stop saying "rationality org". JenniferRM actually brought this up to me once, but I forgot, but there's nothing quite like having seven people agree with a point to make it stick in memory.
It makes me think of "Rationality Orgy", but that's just me. I'm not sure how I feel about that as I haven't been to a meetup yet.
My friend says it makes her think it has to do with boating. There should be a separate focused attempt to come up with the best name.
Sometime ago you believed, correctly IMO, that you need a way of testing rationality skills first, and only then get busy on the exercises. What made you change your mind? (I hope it wasn't something like "we need to push ahead asap".) What's the current plan for preventing the slide into epistemic viciousness? (I hope it isn't something like "we will be smart and won't let it happen".)
We are interested in developing rationality measures; if you have ideas for how to do this, please post; if you're interested in doing larger chunks of work toward developing such measures, please fill in the application form or email me. Blake Riley and I and some other rationality campers worked on this some over the summer, and slow work continues on the same front. Aaron Tucker and I made an experimental daily checklist that we've been playing with, for estimating one's own habits and progress. I'd love to see this work go faster. (I just added a checkbox about this to the application form; thanks for pointing that out; there was a similar item on the call for volunteers that I posted locally some weeks ago, but I forgot about it when posting this round).
It seems to me that rationality measures are valuable, but that creating exercises does not make our present lack of robust measures worse than it already is. Take a look at the linked unit on the sunk costs fallacy, above; when I tested it on newbies (and on LWers), they seemed interested, and started noticing sunk cost fallacy examples in their lives, and did not seem to be much flummoxed by questions of who was how rational or how one could really tell. The sequences already teach some thinking skill without measures (much as the dance class I took in a few years ago helped my dancing some without ever measuring my skill). Measures would be helpful; but refraining from creating exercises until after we have measures does not seem helpful to me.
creating exercises does not make our present lack of robust measures worse than it already is (...) they seemed interested, and started noticing sunk cost fallacy examples in their lives
Martial arts masters and psychotherapy gurus could say the same. Instead of sunk costs you could teach newbies to notice post-colonial alienation or intelligent design, and sure enough they'd get better at noticing that thing in their lives. I hear scientologists do lots of exercises too. Maybe creating exercises before measures is a positive expected value decision, but I wouldn't bet on that.
"Sunk cost" is a pretty well-defined idea, we can reliably figure out whether something is a sunk cost, and whether a decision commits sunk cost fallacy, by checking whether the decision controls the amount of lost value and whether the (immutable) amount of lost value controls the decision. Skill at noticing sunk cost fallacy would then be ability to parse such situations quickly/automatically.
Testing effectiveness of training a skill is easier than testing usefulness of the skill, and I think figuring out how to train people to avoid a list of fallacies or to find correct decisions of standard kinds faster and more reliably is a reasonable goal, even if practical usefulness of having those skills remains uncertain.
The first task of your full-time hire should be coming up with rationality-measuring tools that are better than human intuition.
I would be very, very surprised if you and Anna literally came up with nothing of value on measuring rationality; I expect there's some raw material for a full-time employee to test, tweak and build on. This just seems to me like a higher priority than curriculum-building, and achieving a measure that's better than subjective impressions doesn't even seem impossible to me.
Here's how typical people read typical job ads (typically), especially ones that are this long: Read the title. Scan for a dollar sign or the words "salary" or "salary range". If both are good enough, scan for the first bulleted list of qualifications. Most ads call these "required qualifications". If the reader meets enough of these, they scan for the second bulleted list of qualifications which is usually called "preferred qualifications". Then, if they meet enough of both of these, they'll go back and start reading in detail to understand the position better before they consider sending in an application or contacting the hiring entity for more information.
I suspect that most people expected your job ad to follow this form since it almost does. Your sections are labeled, effectively "needed" and "bonus". It's not until you get to reading the now-bolded details that you find out that not all of the "needed" stuff is required of the applicant and that essentially any one of the needed qualifications will be sufficient. Basically, you don't have any required qualifications, but you do have a general description of the sort of person you're interested in and a list of preferred qualifications. In this regard, the ad is defective as it fails to comport with the usual format of a typical ad.
Non-standard forms get experienced people's hackles up. It often indicates that there's something unprofessional about the organization.
It's a project that has people such as you and lukeprog involved in it. (Luke wasn't mentioned, but he was running the rationality camps etc., so people are going to associate him with this regardless of whether his name is actually mentioned.) You two can, with good reason, be considered Superpeople. I expect that many people will automatically assume that for a cause as important as this, you will only accept folks who are themselves Superpeople as well.
Don't proceed. Stay at the drawing board until you figure out a viable attack. Stay there until you die, if you have to.
If there were a world in which algebra had been learned only through reading essays, without subskill-by-subskill practice, it would not be surprising if the world’s best algebra practitioners could be outperformed by an ordinary student who worked diligently through the exercises in a standard textbook.
This actually happened. The ancient Greeks weren't very capable algebraists because they didn't develop a symbol system that they could systematically manipulate according to prescribed rules. Their description of formal logical inferences were insane to read: "If the first and the second, or the third, implies the fourth, then the first or the fourth, implying the third...." The reason our word "algebra" comes from Arabic isn't because the Muslims were better algebraists; it was because they used symbol systems (to avoid making icons of Mohammad) in order to encode the material they were reading in the Greek literature. The result was something reasonably close to our modern symbol-manipulation system, which made it possible to train in algebra.
So this isn't just a theoretical example. Really, honestly, the first textbook ("al-jebr..." I don't quite remember the title) absolutely trounced several hundred years of careful, intelligent Greek thought on the topic of numerical reasoning.
Edit: Please see this. There's some question about the accuracy of my statement here.
This description is very plausible, but entirely wrong. It was almost completely the opposite of what you're saying. The Muslim mathematicians used fewer symbols than the Greek tradition they inherited for almost the entire timeline of medieval Arabic/Islamic mathematics. The "first textbook" you're referring too, Al-Khwarizmi's Al-jabr wa'l muqabalah, the one ultimately responsible for the word "algebra", did not use any symbols at all, and wrote everything out in words.
Greek mathematicians started to use something like symbols (abbreviated letters with fixed positional meaning) by the time of Diophantus around 3rd century CE. The Arab mathematicians did not adopt that when they translated the Greek texts, and for the first 500 years of their work, wrote everything out fully. Moreover, it is those texts devoid of any symbolic systems that were translated into Latin and used to help fuel the European tradition in 12th-13th centuries CE. Even though some Islamic mathematicians later did develop the beginnings of a symbolic notation, in 14th-15th centuries, this happened roughly in parallel with the Europeans inventing their own symbols, and did not influence the...
To a Greek mathematician, a number was fundamentally a measure of something geometric, like the length of a segment. The square of a number is not some abstract operation: it's just the area of a particular figure, a square. An equation was a way of describing a geometrical problem. Equations were solved geometrically.
Here's an example. Suppose you have unknown numbers x and y, and you know the difference between them and also their product. Can you find x and y? In algebraic terms, you manipulate some unknowns, express y in terms of x and substitute, arrive at a quadratic equation in x, and find the result. Greek mathematicians weren't able to write it this way, in words or in symbols. That just wasn't a way of looking at this problem, or a method of solving it, that they could recognize.
Here's how they thought: you have two unknown lengths. You know by how much one is greater than the other, and you also have a square whose area is equal to the rectangle built on those lengths. Can you find these unknown lengths? Well, you can do it this way: take the difference between them, drawn as a line segment AB. Find its middle point C. Draw a line BQ perpendicular to AB at point B, of l...
Example generation. Given an exercise, we need someone who can think of lots of specific examples from real life or important real-world domains, which illustrate the exact intended point and not something almost-like the intended point. E.g., turn "Sunk cost fallacy" into 20 story snippets like "Lara is playing poker and has bet $200 in previous rounds..." (Our experience shows that this is a key bottleneck in writing a kata, and a surprisingly separate capacity from coming up with the first exercise.)
...so, who else just opened up Evernote and started seeing how many they could bang out?
We're looking for 1-2 fulltime employees who can help us build more things like that (unless the next round of tests shows that the current format doesn't work)
Nice to see you taking to heart the lesson you taught MoR!Harry:
"So what's next?" said Hermione.
Harry rested his head against the bricks. His forehead was starting to hurt where he'd been banging it. "Nothing. I have to go back and design different experiments."
Over the last month, Harry had carefully worked out, in advance, a course of experimentation for them that would have lasted until December.
It would have been a great set of experiments if the very first test had not falsified the basic premise.
Harry could not believe he had been this dumb.
"Let me correct myself," said Harry. "I need to design one new experiment. I'll let you know when we've got it, and we'll do it, and then I'll design the next one. How does that sound?"
"It sounds like someone wasted a whole lot of effort."
Thud. Ow. He'd done that a bit harder than he'd planned.
I'm not sure if this is the right thread for this, but I don't see how example #11 in the "Skill 1: Noticing Sunk Costs" PDF is an actual example of the sunk cost fallacy.
Ellery's options are to eat the slightly-burnt soup, or not to eat and instead to spend $12 (and, presumably, some cooking time) to replace the damaged soup with good soup and eat that instead. She has to eat something (again, presumably); the option to simply abandon the sunk cost, by discarding the soup and then eating nothing, seems implausible. In other words, her decision whether to eat or not eat the soup directly affects her future financial situation, and her decision is based on that effect.
Am I misunderstanding something?
EDIT: Of course it's possible that the negative utility Ellery would get from eating untasty soup outweighs the positive utility from eating at all plus the value of $12, but in any case the question concerns future utility, right?
(My apologies if this is a thread derail.)
Please talk to David McRaney (http://youarenotsosmart.com) to see if he'd be interested. His recent book, while far from comprehensive, has become the first place I look whenever I want to reference an accessible explanation of a particular cognitive bias.
Would it be helpful for us to try out these exercises with a small group of people and report back?
This forum has a pool of people who have many of the talents you requested, but who would not bother applying for one reason or another (unwillingness to overtly commit a significant chunk of time, general akrasia, you name it). However, they would be happy to comment on a single focused question, be it to suggest an activity, to improve a write-up, to do a literature search, or to polish a presentation. I am not sure whether you have anyone on staff who is good at partitioning a project into tiny little tasks of under-an-hour length, but if so, consider an LW section or a post tag [Rationality_org Task] as a means of tapping the low-availability talent pool.
My point is that you might be trying to fill a wrong position.
A qualified part-time volunteer coordinator can do orders of magnitude more good for a non-profit than a full time staff member working on their own. Consider, for example, the VanDusen Botanical Garden. All grounds-keeping and nearly all activities are done by volunteers, with a single coordinator on staff. Some of these volunteer jobs, like the Master Gardener, would be equivalent to probably $50/hr on an open market, maybe more. Some smaller organizations even go one level up, and have a volunteer volunteer coordinator.
Of course, it is harder to properly parcel the jobs in the SI than those in gardening. Then again, none of you in the SI do what you do because you wanted it easy.
In a meeting this morning I suggested that my company was well on its way to needing a development process management suggestion management process manager. Nobody actually threw anything at me, which I attribute to my having been on the phone.
I don't know what others think (besides myself and thomblake, clearly), but I think it's between 3 and 4x under market for a person with those skills in the Bay Area. It's between 2 and 3x under market in a place like Austin, TX, depending on experience.
People with experience doing the things listed above make high 5 and low 6-figure salaries plus benefits (medical, 401k with some matching, etc.) in industry jobs, or they are university or secondary school teachers who have reasonable salaries, health care, and other benefits like tenure not available to industry workers.
It's also possible, for example, that they don't actually want people with work experience doing these things and would settle for folks who are decent at them but have so far only done these activities as a hobby/self-training exercise. If that's the case, then $36k/yr might be OK, and it might be a good opportunity for someone to get these skills on their resume for a later job search in a relevant industry. If that's what they're really looking for, they should state it as such. Otherwise, I remain highly skeptical of the position.
There would not be many people who meet the criteria mentioned that couldn't earn a lot more than that if they wanted it.
You're right, but they don't need many people, they only need one.
(Speaking as someone who applied, has most of those skills pretty solidly (from unusual experiences that employers generally don't care for: professional hula hoop instructor???), but has rarely made more than half of what they are offering)
I have been continuously weirded out by how people in our circles seem to take for granted ridiculous salaries during what's supposed to be an economic recession.
I have been continuously weirded out by how people in our circles seem to take for granted ridiculous salaries during what's supposed to be an economic recession.
This.
Seeing people scoff about how easy it is to make a near six figure income is extremely off-putting.
Keep in mind that SIAI is headquartered in the San Francisco Bay Area, where the cost of living (and thus salaries in general) tend to be higher. I just did a quick Google search and found that in this area, an entry-level police officer can make six figures plus benefits (and eventually pension), so such incomes aren't really outside the realm of normal possibility.
That being said, I think the offered salary is reasonable, especially given the interesting and important nature of the work being done, and will likely apply for the position.
Looking at the sunk cost materials, the answer to question 10 in booklet 1 seems a bit off:
Question: Sandra is starting to grow bored with her current boyfriend. Is it time to move on? She left her job in Los Angeles and moved to New York, just so that she could be with him—but they haven't had a really interesting conversation in months.
Answer: Sunk cost yes, sunk cost fallacy yes.
The time and effort it took to move are already gone and not coming back. Sandra's old job is already gone, also.
Sandra is acting as though her past costs are relevant to her present decision.
I would have marked this as "sunk cost yes, sunk cost fallacy no". The way I read it, Sandra's sunk cost of her leaving her job and moving is mentioned as background information, but it's not stated that Sandra would consider this to be a reason to stay.
One obvious idea for an exercise is MBlume's Positive Bias Test, which is available online.
But of course everyone taking the course would probably already be familiar with the standard example implemented in the app. I would suggest updating the app to have several different patterns, of varying degrees of complexity, and a way for the user to choose the difficulty level before starting the app. I would expect that to be not too hard to implement, and useful enough to be worth implementing.
Shouldn’t the presentation suggest what sort of reasoning is actually going on in someone’s head while they are committing the sunk costs fallacy? For example, (skill 1, problem 12) Peggy is at a baseball game with her son Tim, who is bored and wants to leave. Peggy says, “You want to leave? Those tickets were expensive!”. Her expressed thoughts are a fine example of the sunk costs fallacy. But she may be really thinking: “This is event is the sort of thing normal people pay a lot to attend - Tim must not be thinking clearly. We should stay because he will...
Since a few people have asked me privately: if you're interested in this position, please send in an application even if I already know you. The application is short, and it will let me know that you are interested, and what parts of the work you are interested in and/or able to do.
question 2 sheet 5.
"Answer to 1. Disliked story: Bought a cupboard full of yucky peanut butter. Preferred story: Cleverly purchased good peanut butter in bulk. Fixed reality: You bought a year's supply of a peanut butter you haven't tried, and it's either tasty or yucky; if yucky, pretending to like it and choking it down won't get your money back."
That reality isn't fixed. Tasty/yuckiness is in part determined by your attitude towards the peanut butter. Tastiness is not a one place property. More generally, you can make yourself like different ...
Having seen many concerns about the low salary/skill ratio: it seems like about the same situation that I had for my graduate degree. I took half the salary for a job that was (by my estimate at the time) twice as cool as the alternatives. In that light, the position seems like quite a good deal, as most grad students don't earn that much. If you expect to experience side-benefits from this job, such as benefiting from increased rationality, or having cool stories to tell people about your interesting job, then this seems like a good deal.
Quick rewrite with some bolding to try harder to correct people's impressions that we're trying to hire someone who can do all these things simultaneously.
The formatting is all messed up for me; the text in some later paragraphs is spaced super far apart and runs off the edge of the page. Might be an Opera thing, but I've only seen this on LW, and I've seen it before. Can someone remind me why Markdown is not used for top-level posts?
Here's what it looks like for me.
I would like to have this as an interactive application. It's not always so easy to get into a teacher student situation.
There are applications out there for developing this curriculum in a better way than the regular approach. Like the app these guys are selling http://www.knowledge-values.com/ I use apps like this on a daily basis to create the training content at work.
I don't need a whole game or anything, something as simple as the math exercises on khan academy will do the trick. And i would prefer each skill told as separately and atomically as possible/logically.
The salary for this position seems off by a factor of between 3 and 4 given the sort of background you want. You're asking for someone with professional level design skills coupled to the skills of a university professor or really good high school teacher or video game designer (depending or your perspective). People with these skills get paid a lot more than you're offering. $36k/yr isn't going to get you a bright recent college grad, especially if they have to live in the Bay Area.
It seems to me that you're more interested in hiring folks that are deeply dedicated to the movement so that you can pay them a sub-market salary than hiring the best person you can find. Which is fine, but you should be upfront about it.
The motivation is simply that we need help now, that we do not have budget now, that SingInst's experience suggests that at least some skilled people are willing to work for such money (e.g. me, Carl, Michael Anissimov, Lukeprog), and if rationality org's efforts are successful we will probably have more money for skilled people in the future.
There are several reasons someone might apply, given that. The ones that spring to my mind are:
They're offering 150% of the average US income during a recession with 9% unemployment as starting salary for a potentially entry-level position doing satisfying creative work for an organization that could actually improve the world. I like money as much as anyone else, and I would fight for this job if I weren't otherwise engaged. If my hunt for residency positions this summer falls flat, I might still try to fight for it.
Thanks to the magic of guilds, all new trainee doctor jobs in the US start on July 1st*. If I don't get a job by then, I will probably have to wait until next July and find something to occupy me and provide me with money for a year. Hence my comment that I would be interested if my job search fell flat.
Even though it doesn't look like it sometimes, I do give at least five minutes thought to most of my major life decisions.
*which is why some people have very reasonably argued that you should avoid hospitals at that time of year.
Lots of writers and philosophy postgrads get paid less than this. I don't mean to discourage people with fewer qualifications - a PhD is not required - but we posted a Craigslist ad recently for a different potential position, at a similar salary, and got applications from PhDs with 3 years experience. In any case, we shall see what the market thinks of our offer, and I see no reason for you to take offense at it a priori.
Any idea when you will be choosing candidates? How long should an applicant wait before assuming they were not accepted?
I'm pretty sure there's a mistake in the booklet at link 5 - the preferred story and the disliked story are the wrong way round in the example.
The iPhone app example in the presentation confuses me.
The way it is presented, it looks like the conclusion is that you should always be willing to spend an additional $6999.99 no matter how much you have already spent. If current you is willing to spend the extra money regardless of whether you have already spent $4000 or $10999.99, then I don't see why future you would feel any different.
I would think that you should take in to account the fact that your original estimate of the cost was too low. Given that this is the case, you should expect that ...
Social initiative enough to gather guinea pigs and run many practice trials of draft curriculum, while collecting data.
This seems like something that existing LW meetup groups could do. Perhaps you could send prototype lessons to meetup organizers who would be willing to participate?
Existing LW readers aren't the target audience - the first iteration is aimed at around say the level of the Silicon Valley startup crowd. (Note that you have to imagine yourself aiming lower than this target to successfully hit it.)
I am entirely incapable or many of the things mentioned, I am unable to commit or fully participate due to helth and geographical reasons, and a bunch of other things...
On the other hand I have a feeling feeling of ability to be useful, despite all those things. This kind of deliberate growing of intuitions and making the brain do some specific thing is right up my alley and I might be very good at coming up with certain types of examples or providing a certain kind of hard to articulate meta intuition. I also know a bunch abaut art and design.
I'd also likely find it great fun and far more meaningful than anything I normally do.
So this is a bit of a dillema, what should I do?
From the booklet at link #3:
If you’re going to eat sinful foods, eat high-quality chocolate rather than potato chips—don’t waste calories.
But what if I enjoy potato chips way more than any chocolate ? This piece of advice sounds weird on all kinds of levels...
I just wanted to say that the presentation is wonderful. It's a refreshing change from the usual posts and I would love to see it continue. In fact you should stop the other posts or let them post them on their own blogs or something. This type of content will make a difference in the world, the other guy is more like a well-researched "top 10 ways to lose weight" type of article and has limited tangible benefit.
More please. This is what lesswrong should be about!
Update March 2012: We are still accepting and processing applications for this work on an ongoing basis.
Imagine trying to learn baseball by reading essays about baseball techniques. [1]
We're trying to make the jump to teaching people rationality by, metaphorically speaking, having them throw, catch, and hit baseballs in the company of friends. And as we develop curriculum to do that, we're noticing that we often improve quite a lot ourselves in the course of coming up with 20 examples of the sunk cost fallacy. This suggests that the best of us have a lot to gain from practicing basic skills more systematically. Quoth Anna Salamon:
There are huge numbers of basic, obviously useful rationality habits that I do about 10% as often as it would be useful to do them. Like "run cheap experiments/tests often”, and “notice mental flinches, and track down the thought you’re avoiding”.
Eliezer Yudkowsky, Anna Salamon, several others paid on an hourly basis, and a few volunteers, have been designing exercises and exercise-sets for a rationality curriculum. Our current working point is on the exercises for "Motivated Cognition". Currently the only completed session is "Sunk Costs", which is still being tested - yes, we're actually testing these things repeatedly as we build them. The main purpose of the sessions is to be performed in person, not read online, but nonetheless the current version of the Sunk Costs material - presentation and exercise booklets - is available as a sample: [0] [1] [2] [3] [4] [5]. This is a presentation on sunk costs in which background explanations are interspersed with "do as many of these exercises as you can in 3 minutes", followed by "now pair up with others to do the 'transfer step' parts where you look for instances in your past life and probable future life."
We're looking for 1-2 fulltime employees who can help us build more things like that (unless the next round of tests shows that the current format doesn't work), and possibly a number of hourly contractors (who may be local or distant). We will definitely want to try your work on an hourly or monthly basis before making any full-time hires.
The complete labor for building a rationality kata - we are not looking for someone who can do all of this work at once, we are looking for anyone who can do one or more steps - looks something like this:
Select an important rationality skill and clearly perceive the sort of thinking that goes into executing it. Invent several new exercises which make people's brains execute that type of thinking. Compose many instances of those exercises. Compose any background explanations required for the skills. Figure out three things to tell people to watch out for, or do, over the next week. Turn all of that into a complete 90-minute user experience which includes random cute illustrations for the exercise booklets, designing graphics for any low-level technical points made, building a presentation, testing it in front of a live naive audience, making large changes, and testing it again.
We are not looking only for people who can do all of this labor simultaneously. If you think you can help on one or more of those steps, consider applying — for a full-time job, a part-time hourly gig (perhaps from a distance), or as a volunteer position. We will want anyone hired to try hourly work or a trial month before making any full-time hires. Salary will be SIAI-standard, i.e. $3K/month, but if you do strong work and Rationality-Inst takes off your salary will eventually go much higher. Very strong candidates who can do large amounts of work independently may request higher salaries. You will be working mostly with Anna Salamon and will report to her (although in the short term you may also be working directly with Eliezer on the "isolate a useful skill and invent new exercises to develop it" phase).
If you think you have the idea for a complete rationality kata and want to develop the entire thing on your own, send us a short email about your idea - we're open to setting a lump-sum price.
Skills needed:
We need folks with at least one of the following skills (do not feel you need them all; you'll be part of a team; and repeated experience shows that the people we end up actually hiring, report that they almost didn't contact us because they thought they weren't worthy):
Bonuses:
If this project appeals to you and you think you may have something to add, apply using this short form or just shoot us an email. Please err on the side of applying; so many freaking amazing people have told us that they waited months before applying because they “didn’t want to waste our time”, or didn’t think they were good enough. This project needs many sorts of talents, and volunteers also welcome — so if you’d like to help launch an awesome curriculum, send us an email. Your email doesn’t have to be super-detailed or polished — just tell us how you might be able to contribute, and any experience we should know about.
[1] If the baseball analogy seems far-fetched, consider algebra. To learn algebra, one typically drills one subskill at a time — one spends a day on exponent rules, for example, understanding why x^a * x^b = x^(a+b) and then practicing it bunches of times, in bunches of algebra problems, until it is a part of your problem-solving habits and reflexes, a step you can do fluently while attending to larger puzzles. If there were a world in which algebra had been learned only through reading essays, without subskill-by-subskill practice, it would not be surprising if the world’s best algebra practitioners could be outperformed by an ordinary student who worked diligently through the exercises in a standard textbook. We’d like you to help us build that first textbook.
