Nisan

Wiki Contributions

Comments

What should you change in response to an "emergency"? And AI risk

Taking on a 60-hour/week job to see if you burn out seems unwise to me. Some better plans:

  • Try lots of jobs on lots of teams, to see if there is a job you can work 60 hours/week at.
  • Pay attention to what features of your job are energizing vs. costly. Notice any bad habits that might cause burnout.
  • Become more productive per hour.
Failing to fix a dangerous intersection

Hi Bob, I noticed you have some boxes of stuff stacked up in the laundry room. I can't open the washing machine door all the way because the boxes are in the way. Could you please move them somewhere else?

Dear Alice,

Some of the boxes in that stack belong to my partner Carol, and I'd have to ask her if she's okay with them being moved.

In theory I could ask Carol if she's all right with the idea of moving the boxes. If Carol were to agree to the idea, I would need to find a new place for the boxes, then develop a plan for how to actually move the boxes from one place to another, then get Carol to approve of the plan, then find someone to help me with the bigger boxes, and finally implement the plan.

Though it seems simple enough as an idea, no one would be able to get in or out of the laundry room while I'm maneuvering boxes in there. I would have to coordinate with anyone who wants to do laundry that day to make sure we don't get in each others' way.

Overall, it would be a significant resource-intensive task for me to make and execute such a plan.

I regret I'm unable to proceed any further with your request at this time, as it currently doesn't fit into my to-do list for this week.

I do keep a "someday-maybe" list of projects I can draw from should I ever have some free time, for example if my job unexpectedly gives everyone the day off for some reason.

I already have "empty the lint trap" on this wish list, and will add your suggestion about moving the boxes to the list.

Unfortunately, this is all I can do at this time.

Will working here advance AGI? Help us not destroy the world!

Thanks for sharing your reasoning. For what it's worth, I worked on OpenAI's alignment team for two years and think they do good work :) I can't speak objectively, but I'd be happy to see talented people continue to join their team.

I think they're reducing AI x-risk in expectation because of the alignment research they publish (1 2 3 4). If anyone thinks that research or that kind of research is bad for the world, I'm happy to discuss.

Will working here advance AGI? Help us not destroy the world!

Why do you think the alignment team at OpenAI is contributing on net to AI danger?

Why Go is a Better Game than Chess

Also, chess usually ends in a draw, which is lame. Go rarely if ever ends in a draw.

Shah and Yudkowsky on alignment failures

CFAR used to have an awesome class called "Be specific!" that was mostly about concreteness. Exercises included:

  • Rationalist taboo
  • A group version of rationalist taboo where an instructor holds an everyday object and asks the class to describe it in concrete terms.
  • The Monday-Tuesday game
  • A role-playing game where the instructor plays a management consultant whose advice is impressive-sounding but contentless bullshit, and where the class has to force the consultant to be specific and concrete enough to be either wrong or trivial.
  • People were encouraged to make a habit of saying "can you give an example?" in everyday conversation. I practiced it a lot.

IIRC, Eliezer taught the class in May 2012? He talks about the relevant skills here and here. And then I ran it a few times, and then CFAR dropped it; I don't remember why.

Nisan's Shortform

Agents who model each other can be modeled as programs with access to reflective oracles. I used to think the agents have to use the same oracle. But actually the agents can use different oracles, as long as each oracle can predict all the other oracles. This feels more realistic somehow.

Jimrandomh's Shortform

Ok, I think in the OP you were using the word "secrecy" to refer to a narrower concept than I realized. If I understand correctly, if Alice tells Bob "please don't tell Bob", and then five years later when Alice is dead or definitely no longer interested or it's otherwise clear that there won't be negative consequences, Carol tells Bob, and Alice finds out and doesn't feel betrayed — then you wouldn't call that a "secret". I guess for it to be a "secret" Carol would have to promise to carry it to her grave, even if circumstances changed, or something.

In that case I don't have strong opinions about the OP.

plutonic_form's Shortform

Become unpersuadable by bad arguments. Seek the best arguments both for and against a proposition. And accept that you'll never be epistemically self-sufficient in all domains.

Jimrandomh's Shortform

Suppose Alice has a crush on Bob and wants to sort out her feelings with Carol's help. Is it bad for Alice to inform Carol about the crush on condition of confidentiality?

Load More