All of Qiaochu_Yuan's Comments + Replies

April Coronavirus Open Thread

I'm personally quite worried about disruptions to the food supply chain severe enough to cause food shortages in e.g. the Bay Area in the next few months but not sure what to do with that worry other than to stock up more on non-perishables. Would very much appreciate seeing more people thinking and researching about this.

3BECA2yThe shift of the supply chain from commercial food supply to residential is part of the problem, and is being highlighted right now with producers destroying milk and produce while food banks face severe shortages. These commercial supply chains aren’t easily converted to being direct to consumer, but the government should be purchasing and diverting these excesses to food banks. As individuals, we could all sign up for local CSA veggie boxes, which could contract with commercial suppliers to augment their produce to meet the increased demand. This would reduce collective demand on the direct consumer supply lines (ie: grocery stores). We could also purchase food in bulk from commercial suppliers, and coordinate with neighbors if a single bulk unit is too much to consume before it spoils. The other part of the equation is a potential food shortage in the fall due to farmers being unable to get temporary migrant workers in the spring planting season and projecting that same difficulty for the fall harvest, so they’ve elected to further reduce their spring planting. Unfortunately, I think this has already happened across Europe and North America. [] As individuals, we could all plant subsistence gardens right now to help reduce the collective demand for food in the fall.
We run the Center for Applied Rationality, AMA

FWIW, I don't feel this way about timelines anymore. Lot more pessimistic about estimates being mostly just noise.

Book summary: Unlocking the Emotional Brain
The only part of these processes that actually requires real-time interaction is getting people over what I call their "meta-issue" -- the schema they have that gets in the way of being able to reflect on their issues.
For example, I've had clients who had what you might call a "be a good student" schema that keeps them from accurately reporting their emotions, responses, or progress in applying a reconsolidation technique. Others who would deflect and deny ever having any negative experiences or even any problems, despite having ju
... (read more)

the CFAR techniques as a whole never went meta enough to catch "meta-issues," not in any really systematic way.

There is no level of meta systemization that can overrule a person's meta issues, because their meta issues are always "one level higher than you". ;-) To put it more precisely, no passive information-passing process can bypass a person's meta issues, any more than you can turn a shredder into a fax machine by feeding it a copier manual. The incoming information gets processed through an existing filter that deletes any information that does

... (read more)
Building up to an Internal Family Systems model

Wow, thank you for writing this. This really clarified something for me that I'm in the process of digesting.

Subagents, trauma and rationality

I just got around to reading this; thank you for writing it!

I hadn't thought much about the role of memory in trauma and emotional stuff until pretty recently, possibly based on some kind of present-moment-experience-focused thing I inherited from circling culture. But my experiences using the bio-emotive framework were memory-based in a really important way, and reading this helped something click into place for me about integration being literally integration of memory networks, parts as memory networks, etc.

Using bio-emotive to examine the relati... (read more)

Using bio-emotive to examine the relationship between an emotional reaction I'm having now and a related memory has given the phrase "being present" a meaning it didn't have for me before; often when we aren't present it's because we're in a real sense in the past, possibly way back in the past depending on what memories are being activated.

This reminded me of this bit (not sure if I agree with all of it, but it's an interesting perspective):

The purpose of memory is not to maintain veridical records of the past so
... (read more)
How to find a lost phone with dead battery, using Google Location History Takeout

Google Activity History is sort of terrifying but also great. I used it when someone stole my laptop to learn that the thief had googled pawn shops in the area; I contacted one of the pawn shops they looked up and a bit later they called me telling me someone had brought in a laptop matching my description. They lied to her and told her they needed to process the laptop for a few hours and she needed to come back, and in that time the police were called, she was arrested, and I got my laptop back the same day it was stolen.

The Relationship Between the Village and the Mission

I've been getting a fair number of requests on Facebook for the doc (esp. from community organizers, which I appreciate), and response has been pretty positive. That plus a few other things have me more inclined to write a public draft, but still a little wary of making promises yet.

1Evenflair2yI don't have Facebook but as someone considering deeper participation in the rationalist community, I would greatly appreciate whatever information you have, no matter how messy or disorganized.
The Relationship Between the Village and the Mission

Here is my brain dump: I have mostly given up on the Berkeley rationality community as a possible village. I think the people who showed up here were mostly selected for being bad at villaging, and that the awful shit that's been happening around here lately is downstream of that. I think there is something toxic / dysfunctional woven deep into the community fabric (which has something to do with the ways in which the Mission interacts poorly with people's psychologies) and I don't feel very hopeful about even being able to show it clearly t... (read more)

I've been getting a fair number of requests on Facebook for the doc (esp. from community organizers, which I appreciate), and response has been pretty positive. That plus a few other things have me more inclined to write a public draft, but still a little wary of making promises yet.

6Raemon3y(I have thoughts on this, but probably makes sense to wait to comment here about it until it's at a point where a more fully public conversation is possible. My short take is 'I roughly agree that the dynamics Qiaochu is pointing at are real/important, but I don't think they apply universally – they feel less relevant to me personally, and I think there's something like competing access needs going on.')
2moridinamael3yI'm interested as well. as someone trying to grow the Denver rationality community, I want to be aware of failure modes.
The Hard Work of Translation (Buddhism)

I upvoted this because it gave me some concepts to use to look at some experiences I've had. The speculations at the level of physical mechanism aren't really cruxes for me so I mostly don't care about them, and same with facts of the matter about what any particular Pali text actually says. What's interesting to me is what Romeo gets out of a combination of reading them and reflecting on his own experience, that might be relevant to me reflecting on my own experience.

Why should I believe any of this?

Gut reaction to this question is that it's the wrong question. I don't view this post as telling you anything you're supposed to believe on Romeo's word.

I don’t view this post as telling you anything you’re supposed to believe on Romeo’s word.

On what other basis, then, are we to believe any of this stuff about rewiring neurons, “electrical resistance = emotional resistance”, etc. etc.? We’ve been told that there’s no evidence whatsoever for any of it and that Romeo got the idea for the latter claim, in particular, from literally nowhere at all. So we can’t believe any of this on the basis of evidence, because we’ve been given none, and told that none is forthcoming. And you say we’re not to believe it o

... (read more)
2Elo3yI agree and I want to add that there is a shift away from learning from. The source of an original book, and instead learning from what other people have learnt. And the way they learnt, not just the (very old) original work. Buddhist information is usually participatory. "go and see for yourself" and "don't take my word for it".
How do people become ambitious?

It's goodharting from the point of view of natural selection's values but it doesn't have to be goodharting from the point of view of your values. We can enjoy art even if art is in some sense goodharting on e.g. being in beautiful places or whatever.

The Hard Work of Translation (Buddhism)

This is fantastic and absolutely the conversation I want to be having. Resonates quite a lot with my experience, especially as a description of what it is exactly that I got out of circling.

In your language circling naturally stirs up sankharas because relational shit is happening (e.g. people are paying attention to you or ignoring you, liking or disliking what you say, etc) and then hopefully, if the circle is being well-facilitated, you sometimes get coached into a state where you can notice and work with your "causal links in the perceptual syste... (read more)

How do people become ambitious?

Yeah, I agree that at the earlier stages it's not clear that ambition is a thing to aim for, and I would also advise people to prioritize health broadly.

I agree that encouragement and guidance is good, and more generally think that mentorship is really, really deeply important. I am not about this "individual rationality" life anymore. It's group rationality or nothing.

How do people become ambitious?

Right, this is the kind of thing I had in mind with the phrase "pathological need to do something." Cf. people who are obsessed with making way more money than they could ever possibly spend.

Has "politics is the mind-killer" been a mind-killer?
He says that "Politics is an extension of war by other means. Arguments are soldiers". I'd update this to say that "ONE WAY OF THINKING ABOUT politics is AS IF IT IS an extension of war by other means. Arguments CAN BE THOUGHT OF AS soldiers". 

This is a good shift for you to have made and I'm glad you can make it. Now you can just do this mentally to everyone's writing (and speaking, for that matter) all the time.

But asking writers to do it themselves is crippling. The new sentence you propose is obviously technically mo... (read more)

Open Thread April 2019

Terence Tao is great; I haven't read that book but I like his writing a lot in general. I am a big fan of the Princeton Lectures in Analysis by Stein and Shakarchi; clear writing, good exercises, great diagrams, focus on examples and applications.

(Edit: also, fun fact, Stein was Tao's advisor.)

Epistemic status: ex-math grad student

How do people become ambitious?

I think you're equivocating between two possible meanings of "choose" here. There's "choose" as in you start telling people "I want to write a book" and then there's "choose" as in you actually decide to actually write the book, which is quite different. I think Ray is asking about something like how to cultivate the capacity to do the latter. It is not at all trivially easy. Most goals are fake; making them real is a genuine skill.

1moses3yAh, yes, when I say "choose", I mean system-2-choose (i.e. the former meaning in your comment). Learning how to (i.e. how to work with setting intentions, or in general, how to overcome akrasia) would already be a to-do included on the big to-do list called "achieving goal X". In any case, if I understand it correctly, the question still is: how do people become capable of achieving big goals, including whatever system-1 manipulation, intention-setting, habit-forming, incentive-landscape-shaping, motivation-hacking, etc. is necessary to achieve these goals?
How do people become ambitious?

Tongue-in-cheek: "when their pathological need to do something outweighs their pathological need to do nothing."

In more detail: there are several different kinds of deep-rooted psychological needs that ambition might be powered by, and I think the resulting different kinds of ambition are different enough to discuss as distinct entities (in particular, they vary in how prosocial they are). Some possibilities off the top of my head, not mutually exclusive, inspired by Enneagram types:

1. Reinforce a particular identity / self-narrative (e.g. &quo... (read more)

3Raemon3yThanks. This is a different lens than my pet theory but seems pretty compatible with it. My pet theory was specifically "one contributor towards people becoming ambitious is having someone they respect tell them that they believe in them (credibly signaling it by spending at least a few hours talking to them about their projects and goals and how to achieve them)." This is based on what happened to me and a few other people I know who gained certain kinds of ambition. [This comes with some background beliefs that ambition is generally good, in particular when the ambition is of the form 'create something', rather than 'be the best at something' (which is zero-sum). This may be a different lens than you're currently looking through] This seems somewhat downstream of the sort of thing you're point at here, which seems to be pointing at two clusters: * What sort of personality do you need to have for "become ambitious" to be a live option (including healthy and unhealthy mechanisms) * What sort of things could actively harm your capacity for ambition. Both of which suggest much earlier interventions than what I was thinking about. I basically want all the interventions going on, but at the earlier stages, it's not obvious that the right thing for a person is to become ambitious. Rather, I want them to have a psychologically outlook that lets them be generally physically and emotionally healthy and reasonably productive (because these are generally good things). Once they get to that stage, maybe the right thing for them is to tackle ambitious projects, maybe not. But it seems better to think of interventions at that stage to be more oriented towards 'help the person become generally healthy' rather than 'help them become ambitious.' For people just reaching the stage where they're considering an ambitious project (either one that'll directly accomplish something, or help them grow) I think having a someone give them a few hours of encouragement and guidan

I also don't have much of this skill and made it through life without needing to have it; I was able to coast on raw intelligence for quite a long time, up through my 2nd year of grad school or so. Welp.

Except in romantic relationships; I've historically consistently found it easy to have commitment, follow-through, reliability, focused attention, etc. in that context (although it was kinda being fueled by neediness so there were other things going on there).

It feels like I have not yet found e.g. a job that I deeply value enough to commit to in... (read more)

What Vibing Feels Like

Yes, I strongly agree that this is missing and it sucks. I have a lot to say about why I think this is happening, which hopefully will be converted from a Google Doc into a series of blog posts soonish.

1Matt Goldenberg3yLooking forward to this. Feel free to send me an invite to look over the google doc.
Policy-Based vs Willpower-Based Intentions

There's an interesting thing authentic relating people do at workshops that they call "setting intentions," and I think it works in a different way than either of these. The difference seems to me to be that the intention is being "held by the group." I'm not sure how to explain what I mean by this. There are at least two visible signs of it:

1) people remind each other about the intention, and

2) people reward each other for following through on the intention.

If everyone knows the intention is being held by the group in this... (read more)

7Unreal3yI have not much considered group intention-setting. This seems super interesting to explore too. Phenomenologically, I feel it kind of as... the agreements or intentions of the group (in a circle) recede into the background, to form the water we're all in together. Like it gets to relax in the VERY BACK of my mind and also I'm aware of it being in the back of other people's minds. And from that shared container / background, I "get to move around" but it's like I am STARTING with a particular set of assumptions. Other potential related examples: * I'm at a Magic tournament. I know basically what to expect—what people's goals are, what people's behaviors will be, what the rules of the game are and how to enforce them. It's very easy for me to move here because a lot of the assumptions are set in place for me. * I'm in church as a kid. Similar to the above. But maybe less agreeable to me or more opaque to me. I get this weird SENSE that there are ways I'm supposed to behave, but I'm not totally sure what they are. I'm just trying to do what everyone else seems to be doing... This is not super comfortable. If I act out of line, a grownup scolds me, is one way I know where the lines are. Potential examples of group policy-based intentions: * I have a friend I regularly get meals with. We agree to take turns paying for each other, explicitly. * I have a friend, and our implicit policy is to tell each other as soon as something big happens in our lives. As soon as a third person is added to the dynamic, I think it gets trickier to ensure it's a policy-based intention. (Technology might provide many exceptions?) As soon as one person feels a need to remind themselves of the thing, it stops being a policy-based intention. Willpower-based intentions in groups feel they contain a bunch of things like rules, social norms, etc.
3Unreal3yThere is definitely this sense that exerting force or willpower feels like an EXTERNAL pressure even if that pressure does not have an external source that I could point to or even gesture at. But it /feels/ external or 'not me'. I have some trauma related to this. I could've gone into the trauma stuff more, but I think it would have made the post less accessible and also more confusing, rather than less. So I didn't. :P
Informal Post on Motivation

Glad to see you're writing about this! I think motivation is a really central topic and there's lots more to be said about it than has been said so far around here.

When we're struggling with motivation to do X, it's because only S2 predicts/believes that X will lead to reward. S1 isn't convinced. Your S2 models say X is good, but S1 models don't see it. This isn't necessarily a bug. S2 reasoning can be really shitty, people can come up with all kinds of dumb plans for things that won't help, and it's not so bad
... (read more)
Epistemic Tenure

This seems like a bad idea to me; I think people who are trying to have good ideas should develop courage instead. If you don't have courage your ideas are being pushed around by fear in general, and asking for a particular source of that fear to be ameliorated will not solve the general problem.

Two Small Experiments on GPT-2

Thanks for writing this up! I'm excited to see more people running experiments like this.

When you say "if I take X as a prompt, I get Y," how many trials did you wait? In my own experimentation I've found lil' GPT-2's performance to be really variable across trials, and I've needed to wait 5 trials in some cases to get results I even sort of liked.

My sense overall of how lil' GPT-2 functions after playing with it for awhile on several different kinds of prompts is that it has a strong sense of genre, and has done so... (read more)

3Gurkenglas3yTry varying lines 14 and 16 in the interactive script for quicker execution, and try giving it a few example lines to start with.
6jimrandomh3yThis definitely could use more trials. In the case of the sentiment analysis experiment, I'd ideally like to try out some other sentence structures (eg "Is a <noun> bad?", "Are <adjective> things good?); in the case of the Moloch experiment, I'd like to try some reruns with the same parameters, as well as different name substitutions, just to be sure that it isn't noise.
Building up to an Internal Family Systems model

Thanks for writing this! I am very excited that this post exists. I think what this model suggests about procrastination and addiction alone (namely, that they're things that managers and firefighters are doing to protect exiles) are already huge, and resonate strongly with my experience.

In the beginning of 2018 I experienced a dramatic shift that I still don't quite understand; my sense of it at the time was that there was this crippling fear / shame that had been preventing me from doing almost anything, that suddenly lifted (for several reaso... (read more)

things that had been too scary for me to think about became thinkable (e.g. regrettable dynamics in my romantic relationships), and I think this is a crucial observation for the rationality project. When you have exile-manager-firefighter dynamics going on and you don't know how to unblend from them, you cannot think clearly about anything that triggers the exile, and trying to make yourself do it anyway will generate tremendous internal resistance in one form or another (getting angry, getting bored, getting sleepy, getting confused, all sorts of cr
... (read more)
Transhumanism as Simplified Humanism

I like this reading and don't have much of an objection to it.

5Rob Bensinger3yK, cool. :)
Transhumanism as Simplified Humanism

This is a bad argument for transhumanism; it proves way too much. I'm a little surprised that this needs to be said.

Consider: "having food is good. Having more and tastier food is better. This is common sense. Transfoodism is the philosophy that we should take this common sense seriously, and have as much food as possible, as tasty as we can make it, even if doing so involves strange new technology." But we tried that, and what happened was obesity, addiction, terrible things happening to our gut flora, etc. It is just blatantly false in ge... (read more)

Yeah, "Life is good" doesn't validly imply "Living forever is good". There can obviously be offsetting costs; I think it's good to point this out, so we don't confuse "there's a presumption of evidence for (transhumanist intervention blah)" with "there's an ironclad argument against any possible offsetting risks/costs turning up in the future".

Like Said, I took Eliezer to just be saying "there's no currently obvious reason to think that the optimal healthy lifespan for most people i... (read more)

Consider: “having food is good. Having more and tastier food is better. This is common sense. Transfoodism is the philosophy that we should take this common sense seriously, and have as much food as possible, as tasty as we can make it, even if doing so involves strange new technology.” But we tried that, and what happened was obesity, addiction, terrible things happening to our gut flora, etc. It is just blatantly false in general that having more of a good thing is better.

Conclusion does not follow from example.

You are making exactly the mistake which

... (read more)
Preliminary thoughts on moral weight

This whole conversation makes me deeply uncomfortable. I expect to strongly disagree at pretty low levels with almost anyone else trying to have this conversation, I don't know how to resolve those disagreements, and meanwhile I worry about people seriously advocating for positions that seem deeply confused to me and those positions spreading memetically.

For example: why do people think consciousness has anything to do with moral weight?

why do people think consciousness has anything to do with moral weight?

One of my strongest moral intuitions is that suffering is bad, meaning that it's good to help other minds not-suffer. Minds can only suffer if they are conscious.

why do people think consciousness has anything to do with moral weight?

Is there anything that it seems to you likely does have to do with moral weight?

I feel pretty confused about these topics, but it's hard for me to imagine that conscious experience wouldn't at least be an input into judgments I would endorse about what's valuable.

For anyone who is curious, I cite much of the literature arguing over criteria for moral patienthood/weight in the footnotes of this section of my original moral patienthood report. My brief comments on why I've focused on consciousness thus far are here.

Relevant reading: gwern's The Narrowing Circle. He makes the important point that moral circles have actually narrowed in various ways, and also that it never feels that way because the things outside the circle don't seem to matter anymore. Two straightforward examples are gods and our dead ancestors.

4habryka3yGreat article, can strongly recommend. Hadn't read it before and got quite a bit of value out of it.
Open Thread August 2018

Does anyone else get the sense that it feels vaguely low-status to post in open threads? If so I don't really know what to do about this.

2ChristianKl3yI don't think it's a problem even if it's true. It's better when people feel drawn to write more thought out post because they perceive that as higher status.
3gjm3yFor what it's worth, I've never felt that -- but I've not posted a lot of actual posts, and maybe if that were my point of comparison (as opposed to commenting elsewhere) I might feel differently.
2habryka3yI have a similar sense, used to kinda endorse it, but now think I was wrong and would like to fix it.
3Dagon3yWith the personal-blog category of posts, I think there's a lot lower barrier to the main site, so less need for the open threads. I don't think that translates to low-status, but I'm a bad indicator of status - I've been here for over a decade and never made a top-level post. I'm just a comment kind of guy, I guess.
Strategies for Personal Growth

This makes sense, but I also want to register that I viscerally dislike "controlling the elephant" as a frame, in roughly the same way as I viscerally dislike "controlling children" as a frame.

Strategies for Personal Growth

Huh. Can you go into more detail about what you've done and how it's helped you? Real curious.

Not mr-hire, but I got a lot of value from things more in the space of "improve your ability to control the elephant." I expect the lowest hanging fruit will vary from person to person, and part of the point of this post was to alert people to the fact that there are different strategies that they might not be considering (either for themselves or people they're giving advice to)

I think there was a general skill of "learn how to focus on a task", which I learned by combination of:

– finding projects that I cared enough about to actu... (read more)

Strategies for Personal Growth
I think the original mythology of the rationality community is based around cheat codes

A lot of the original mythology, in the sense of the things Eliezer wrote about in the sequences, is about avoiding self-deception. I continue to think this is very important but think the writing in the Sequences doesn't do a good job of teaching it.

The main issue I see with the cheat code / munchkin philosophy as it actually played out on LW is that it involved a lot of stuff I would describe as tricking yourself or the rider fighting against / overriding the ele... (read more)

As a counter to this, I got very very far with this sort of self-improvement for a very long time (though I think LW was very bad at teaching it, and I mostly got it from other sources.) I've recently focused on the alignment based models as I was starting to get to the point of diminishing returns with the other way, but I did get a lot out of the previous paradigm

I think the alignment based models are very very powerful, and I also think that the overriding the elephant models are quite powerful and get too much of a bad rap.

ISO: Name of Problem

This question feels confused to me but I'm having some difficulty precisely describing the nature of the confusion. When a human programmer sets up an IRL problem they get to choose what the domain of the reward function is. If the reward function is, for example, a function of the pixels of a video frame, IRL (hopefully) learns which video frames human drivers appear to prefer and which they don't, based on which such preferences best reproduce driving data.

You might imagine that with unrealistic amounts of computational power IRL might attempt... (read more)

2johnswentworth3yProblem is, if there's a sufficiently large amount of sufficiently precise data, then the physically-correct model's high accuracy is going to swamp the complexity penalty. That would be a ridiculously huge amount of data for atom-level physics, but there could be other abstraction levels which require less data but are still not what we want (e.g. gene-level reward functions, though that doesn't fit the driving example very well). Also, reliance on limited data seems like the sort of thing which is A Bad Idea for friendly AGI purposes.
ISO: Name of Problem

IRL does not need to answer this question along the way to solving the problem it's designed to solve. Consider, for example, using IRL for autonomous driving. The input is a bunch of human-generated driving data, for example video from inside a car as a human drives it or more abstract (time, position, etc.) data tracking the car over time, and IRL attempts to learn a reward function which produces a policy which produces driving data that mimics its input data. At no point in this process does IRL need to do anything like reason about the distinctio... (read more)

8jessicata3yImportantly, this only works for narrow value learning [], not what Paul calls "ambitious value learning" (learning long-term preferences). Narrow value learning has much more in common with imitation learning than ambitious value learning; at best, you end up with something that pursues similar subgoals to the ones humans do. The concern in the original post applies to ambitious value learning. (But ambitious value learning using IRL already looks pretty doomed [] anyway).
3johnswentworth3yWouldn't the reward function "maximize action for this configuration of atoms" fit the data really well (given unrealistic computational power), but produce unhelpful prescriptions for behavior outside the training set? I'm not seeing how IRL dodges the problem, other than the human manipulating the algorithm (effectively choosing a prior).
Replace yourself first if you're moving to the Bay

Thanks for the mirror! My recommendation is more complicated than this, and I'm not sure how to describe it succinctly. I think there is a skill you can learn through practices like circling which is something like getting in direct emotional contact with a group, as distinct from (but related to) getting in direct emotional contact with the individual humans in that group. From there you have a basis for asking yourself questions like, how healthy is this group? How will the health of the group change if you remove this member from it? Etc.

It also s... (read more)


I think there is something interesting here. I can see how I might be missing a perspective or modality that allows you to model small groups of people more directly, which is something I sometimes get hints of, but usually don't do too much (i.e. I don't participate in lots of group flow-states, communal dancing, most forms of circling, etc.). I could see how you could use that perspective to build high-level models of a community, though I am still not sure whether that's actually a good idea. But it seems worth a try.

Replace yourself first if you're moving to the Bay

I appreciate the thought. I don't feel like I've laid out my position in very much detail so I'm not at all convinced that you've accurately understood it. Can you mirror back to me what you think my position is? (Edit: I guess I really want you to pass my ITT which is a somewhat bigger ask.)

In particular, when I say "real, living, breathing entity" I did not mean to imply a human entity; groups are their own sorts of entities and need to be understood on their own terms, but I think it does not even occur to many people to try in the sense that I have in mind.

Sure, let me give it a try:

My model is that you are recommending to think about communities in a way that I would describe as "using your emotional modalities" and encouraging people to try to connect the consequences of their actions, to have a direct emotional impact on their experience. The most straightforward way to do this, is to try to increase the level of empathy you have for the people in the community. One way I would expect one could achieve that concretely, is by imagining you taking an action, such as moving to the Bay Area, and th... (read more)

Replace yourself first if you're moving to the Bay

(For additional context on this comment you can read this FB status of mine about tribes.)

There's something strange about the way in which many of us were trained to accept as normal that two of the biggest transitions in our lives - high school to college, college to a job - get packaged in with abandoning a community. In both of those cases it's not as bad as it could be because everyone is sort of abandoning the community at the same time, but it still normalizes the thing in a way that bugs me.

There's a similar normalization of abandonme... (read more)

I thought about this comment for an hour after I read it, and think this perspective is actively bad advice for community/institution building, at least applied naively, and I also expect applied at the level at which you are thinking about it. I think mentally anthropomorphizing communities of our size (i.e. anything above ~100) gives rise to really bad heuristic, and I think will cause you to waste the waste majority of your effort, and get critical questions like the ones discussed in this article wrong. I expect starting to do it will initially give yo... (read more)

Yes, absolutely. This is what graduate school and CFAR workshops are for. I used to say both of the following things back in 2013-2014:

  • that nearly all of the value of CFAR workshops came from absorbing habits of thought from the instructors (I think this less now, the curriculum's gotten a lot stronger), and
  • that the most powerful rationality technique was moving to Berkeley (I sort of still think this but now I expect Zvi to get mad at me for saying it).

I have personally benefited a ton over the last year and a half through osmosing things from dif... (read more)

A framework for thinking about wireheading

I think your description of the human relationship to heroin is just wrong. First of all, lots of people in fact do heroin. Second, heroin generates reward but not necessarily long-term reward; kids are taught in school about addiction, tolerance, and other sorts of bad things that might happen to you in the long run (including social disapproval, which I bet is a much more important reason than you're modeling) if you do too much heroin.

Video games are to my mind a much clearer example of wireheading in humans, especially the ones furthest in the fa... (read more)

Bayesian Probability is for things that are Space-like Separated from You
In particular, you shouldn't force yourself to believe that you're attractive.

And I never said this.

But there's a thing that can happen when someone else gaslights you into believing that you're unattractive, which makes it true, and you might be interested in undoing that damage, for example.

Bayesian Probability is for things that are Space-like Separated from You

Yes, this.

There's a thing MIRI people talk about, about the distinction between "cartesian" and "naturalized" agents: a cartesian agent is something like AIXI that has a "cartesian boundary" separating itself from the environment, so it can try to have accurate beliefs about the environment, then try to take the best actions on the environment given those beliefs. But a naturalized agent, which is what we actually are and what any AI we build actually is, is part of the environment; there is no cartesian boundary. Among ... (read more)

Bayesian Probability is for things that are Space-like Separated from You
The problem is that the standard justifications of Bayesian probability are in a framework where the facts that you are uncertain about are not in any way affected by whether or not you believe them!

I want to point out that this is not an esoteric abstract problem but a concrete issue that actual humans face all the time. There's a large class of propositions whose truth value is heavily affected by how much you believe (and by "believe" I mean "alieve") them - e.g. propositions about yourself like "I am confident" or eve... (read more)

5cousin_it4yI'm confused by this. Sure, your body has involuntary mechanisms that truthfully signal your beliefs to others. But the only reason these mechanisms could exist is to help your genes! Yours specifically! That means you shouldn't try to override them when your interests coincide with those of your genes. In particular, you shouldn't force yourself to believe that you're attractive. Am I missing something?
I think the LW zeitgeist doesn't really engage with this.

Really? I feel quite the opposite, unless you're saying we could do still more. I think LW is actually one of the few communities that take this sort of non-dualism/naturalism in arriving at a probabilistic judgement (and all its meta levels) seriously. We've been repeatedly exposed to the fact that Newcomblike problems are everywhere since a long time ago, and then relatively recently, with Simler's wonderful post on crony beliefs (and now, his even more delightful book with Ha... (read more)

An Exercise in Applied Rationality: A New Apartment

I do not. Fortunately, you can just test it empirically for yourself!

An Exercise in Applied Rationality: A New Apartment

General advice that I think basically applies to everybody is to try to lock down sleep, diet, and exercise (not sure what order these go in exactly) solidly.

Random sleep tips:

  • Try to sleep in as much darkness as possible. Blackout curtains + a sleep mask is as dark as I know how to easily make things, although you might find the sleep mask takes some getting used to. Just a sleep mask is already pretty good.
  • Blue light from screens at night disrupts your sleep; use f.lux or equivalent to straightforwardly deal with this.
  • Lower body temperature makes it ea
... (read more)
2ChristianKl4yThat's interesting advice I haven't heard before. Do you have further references to know more about the size of that effect?
Stories of Summer Solstice

The drum circle leading up to sunset was beautiful, but the drum (+ dance + singing) circle after sunset was really fun. I drummed and it was fun! Then I danced and it was fun! Then I sang and it was fun! Nat started improvising a melody and I tried that and it was fun, and then Nat started improvising lyrics and I tried that and it was even better

and then we played one of my favorite games, sing-as-many-songs-with-the-same-chord-progression-at-the-same-time-as-possible, with the most people I've ever gotten to do it with –

anyway, all of that made me really happy, and I feel really grateful to everyone who helped make it all possible.

Whoops, hang on, I definitely did not intend for all of these posts to be crossposted to LW. I thought Ben Pace had set things up so that only things tagged #lw would be crossposted, but that doesn't seem to have been what happened. My bad. I can't seem to delete the posts or make them invisible.

Last Chance to Fund the Berkeley REACH

Pledged $50 / mo. I haven't been to an event at REACH yet but I'm happy about the events I've seen on Facebook being hosted there, and expect to attend and/or host something there in the nearish future if it keeps existing. Everything Ray et al. have been writing about community health and so forth resonates with me and I'm happy to put my money where my resonance is.

Why kids stop asking why

I think a pattern that makes sense to me is cycles of exploration and exploitation: learn about the world, act on that understanding, use the observations you acquired from acting to guide further learning, etc. The world is big and complicated enough that I think you don't hit anything close to diminishing marginal returns on asking "why?" (my experience, if anything, has been increasing marginal returns as I've gotten better at learning things), although I agree that it's important to get some acting going on in there too.

8Viliam4yI agree, but this probably wasn't true in our ancient evolutionary environment.
Meta-Honesty: Firming Up Honesty Around Its Edge-Cases

Benquo, this is really great to hear. This is a shift I went through gradually throughout 2017 and I think it's really important.

Teaching Methodologies & Techniques

Teaching is not about methodology; it's metis, not episteme. (I am also not a schoolteacher but I have taught at CFAR workshops.)

I love cousin_it's suggestion that you should start teaching a student regularly as soon as possible, but I have an additional suggestion about how to spend that time: namely, your goal should not be to teach anyone anything but to find out how students' minds work (and since anyone can be a student, this means your goal is to find out how people's minds work), and how those minds interface with the material ... (read more)

Load More