cata

Programmer, rationalist, chess player, father, altruist.

Comments

I think few of us in the alignment community are actually in a position to change our minds about whether alignment is worth working on. With a p(doom) of ~35% I think it's unlikely that arguments alone push me below the ~5% threshold where working on AI misuse, biosecurity, etc. become competitive with alignment. And there are people with p(doom) of >85%.

This makes little sense to me, since "what should I do" isn't a function of p(doom). It's a function of both p(doom) and your inclinations, opportunities, and comparative advantages. There should be many people for whom, rationally speaking, a difference between 35% and 34% should change their ideal behavior.

Answer by cataMar 16, 2023135

Since there's a very broad spectrum of different kinds of computer programs with different constraints and desiderata, I think the transition will be very gradual. Consider the following things that are all computer programming tasks:

  • Helping non-technical people set up a simple blog.
  • Identifying and understanding the cause of unexpected behavior in a large, complicated existing system.
  • Figuring out how to make a much cheaper-to-run version of an existing system that uses too many resources.
  • Experimenting with a graphics shader in a game to see how you can make an effect that is really cool looking.
  • Implementing a specific known cryptographic algorithm securely.
  • Writing exploratory programs that answer questions about some data set to help you understand patterns in the data.

I have no doubt that sufficiently fancy AI can do or help human programmers do all these tasks, but probably in different ways and at different rates.

As an experienced programmer that can do most of these things well, I would be very surprised if my skillset were substantially obsolete in less than 5 years, and somewhat surprised if it was substantially obsolete in less than 10 years. It seems like GPT-3 and GPT-4 are not really very close to being able to do these things as well as me, or close to being able to help a less skilled human do these things as well as me.

As a LW veteran interested in EA I also perceive a lot of the dynamics you wrote about and they really bother me. Thank you for your hard and thoughtful work.

Answer by cataMar 10, 202320

It seems to me that the world into which children are born today has a high likelihood of being really bad.

Why? You didn't elaborate on this claim.

I would certainly be willing to have a kid today (modulo all the mundane difficulty of being a parent) if I was absolutely, 100%, sure that they would have a painful death in 30 years. Your moral intuitions may vary. But have you considered that it's really good to have a fun life for 30 years?

When I was in middle school and high school (Michigan, 1996-2004) the only identifiable political engagement I remember any of my peers doing ever was that we knew it was funny to make fun of George Bush for being an idiot. I don't remember ever having any other conversation with my friends about any political topic. I certainly had no clue what was going on in politics, outside of knowing who the president was, and knowing that 9/11 happened, and knowing that the Iraq War existed. So the idea that teenagers have political opinions now is also striking to me.

Maybe this is just quibbling about words, but I don't like conflating "weird" with "uncharismatic", "awkward", "disagreeable", "picky", etc. There are many very weird (unusual; people will be surprised if you do them; you will stand out) behaviors that don't result in the kinds of drawbacks you listed, e.g. signing up for cryonics, or doing moral reasoning from first principles, or having polyamorous relationships.

They only have those drawbacks if you then do things like discuss them tactlessly, or make them central to your social identity, or act judgmentally or intolerantly towards people who behave normally, etc. But you can usually just not do those things.

I'm not a machine learning researcher, but this is fascinating and I can't wait to see what else you can dig up about this phenomenon!

But cata, where does your "stuff that seems like it would be a good idea to do right now" queue come from? If you cannot see its origin, why do you trust that it arises primarily from your true values?

Well, I trust that because at the end of the day I feel happy and fulfilled, so they can't be too far off.

I believe you that many people need to see the things that are invisible to them, that just isn't my personal life story.

What you say makes sense. I think most of the people "doing whatever it is that people do" are making a mistake.

The connection to "masking" is very interesting to me. I don't know much about autism so I don't have much background about this. I think that almost everyone experiences this pressure towards acting normal, but it makes sense that it especially stands out as a unique phenomenon ("masking") when the person doing it is very not-normal. Similarly, it's interesting that you identify "independence" as a very culturally-pushed value. I can totally see what you mean, but I never thought about it very much, which on reflection is obviously just because I don't have a hard time being "the culturally normal amount of independent", so it never became a problem for me. I can see that the effect of the shared culture in these cases is totally qualitatively different depending on where a person is relative to it.

One of the few large psychological interventions I ever consciously did on myself was in about 2014 when I went to one of the early CFAR weekend workshops in some little rented house around Santa Cruz. At the end of the workshop there was a kind of party, and one of the activities at the party was to write down some thing you were going to do differently going forward.

I thought about it and I figured that I should basically stop trying to be normal (which is something that before I thought was actively virtuous, for reasons that are now fuzzy to me, and would consciously try to do -- not that I successfully was super normal, but I was aiming in that direction.) It seemed like the ROI on being normal was just crappy in general and I had had enough of it. So that's what I did.

It's interesting to me that some people would have trouble with the "how to live more authentically instead" part. My moment to moment life feels like, there is a "stuff that seems like it would be a good idea to do right now" queue that is automatically in my head, and I am just grabbing some things out of it and doing them. So to me, the main thing seems to be eliminating any really dumb biases making me do things I don't value at all, like being normal, and then "living more authentically" is what's left.

But that's just my way -- it would make sense to me if other people behaved more strategically more often, in which case I guess they might need to do a lot more introspection about their positive values to make that work.

I want to say something, but I'm not really sure how to phrase it very precisely, but I will just say the gist of it in some rambly way. Note: I am very much on the periphery of the phenomenon I am trying to describe, so I might not be right about it.

Most EAs come from a kind of western elite culture that right now assigns a lot of prestige to, like, being seen to be doing Important Work with lots of Power and Responsibility and Great Meaning, both professionally and socially.

"I am devoting my life to solving the most important problems in the world and alleviating as much suffering as possible" fits right into the script. That's exactly the kind of thing you are supposed to be thinking. If you frame your life like that, you will fit in and everyone will understand and respect what is your basic deal.

"I am going to have a pleasant balance of all my desires, not working all that hard, spending some time on EA stuff, and the rest of the time enjoy life, hang out, read some books, and go climbing" does not fit into the script. That's not something that anyone ever told you to do, and if you tell people you are going to do that, they will be surprised at what you said. You will stand out in a weird way.

Example anecdote: A few years ago my wife and I had a kid while I was employed full-time at a big software company that pays well. I had multiple discussions roughly like this with my coworkers:

  • Me: My kid's going to be born this fall, so I'll be taking paternity leave, and it's quite likely I will quit after, so I want to figure out what to do with this thing that I am responsible for.
  • Them: What do you mean, you will quit after?
  • Me: I mean I am going to have a baby, and like you, they paid me lots of money, so my guess is that I will just hang out being a parent with my wife and we can live off savings for a while.
  • Them: Well, you don't have to do that! You can just keep working.
  • Me: But doesn't it sound like if you were ever going to not work, the precise best time would be right when you have your first kid? Like, that would be literally the most common sense time in your life to choose not to work, and pay attention to learning about being a parent instead? I can just work again later.
  • Them: [puzzled] Well, you'll see what I mean. I don't think you will quit.

And then they were legitimately surprised when I quit after paternity leave, because it's unusual for someone to do that (at least not men) regardless of whether they have saved a bunch of money due to being a programmer. The normal thing to do is, let your work define your role in life and gives you all your social capital, so it's basically your number 1 priority, and everything else is a sideshow.

So it makes total sense to me that EAs who came from this culture decide that EA should define their role in life and give them all their social capital and be their number 1 priority, and it's not about a failure of introspection, or about a conscious assessment of their terminal values that turned out wrong. It's just the thing people do.

My prediction would be that among EAs who don't come from a culture with this kind of social pressure, burnout isn't really an issue.

Load More