evhub's Shortform

evhub

This is a special post for quick takes by evhub. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I think more people should seriously consider applying to the Anthropic Fellows program, which is our safety-focused mentorship program (similar to the also great MATS). Applications close in one week (August 17). I often think of these sorts of programs as being primarily useful for the skilling up value they provide to their participants, but I've actually been really impressed by the quality of the research output as well. A great recent example was Subliminal Learning, which was I think a phenomenal piece of research that came out of that program and was jointly supervised by Sam Marks (at Anthropic) and Owain Evans (not at Anthropic—a little-known fact is that we also have non-Anthropic mentors!). That's also not to say that it hasn't been great for skilling people up as well—alongside MATS (which is also similarly great!), Anthropic Fellows is probably our main recruiting pipeline for safety roles right now (so if you want to do safety research at Anthropic, Anthropic Fellows is very likely your best bet!). The program is also very open to people with a really wide range of degrees of prior experience—we take in some people with a ton of prior experience, and some people with basically none at all—so it's an especially great on ramp for new people, and also at the same time one of the best ways for more experienced people to effectively show that they have the requisite skills.

fwiw, from my time at MATS, I recall several projects that were ‘just capabilities’. I’m not sure if those ended up being published on, or what the overall ratio was (it wasn’t 1/6, surely, but the Anthropic program also has a much smaller sample size than MATS).

To the extent that some people updated on the fellows program based on this comment, it’s likely they should also update on MATS (although to a lesser degree), and I’d be interested in an analysis of MATS research outputs that found the ratio (maybe they’ve already done this analysis, and maybe the... (read more)

4Ben Pace1y

I admit I'm not sure how to read this as something other than a rejection of holding people morally accountable.

2the gears to ascension1y

this is getting off topic, but I'll give one more reply: "holding accountable" in the sense of "sending them frowny faces to make them feel bad about their behavior" doesn't seem like a consequentially effective frame, yeah. if someone is doing bad things, you take actions to stop them; if they're convinceable and just aren't considering that the things might be bad, you show them the impact, and then they feel bad when they comprehend. if they're not convinceable, you aren't going to get through to them by frowning at them. claiming moral failure is a "send you frowny face" move. like - let's say you bumped into someone on a crowded subway car, because you weren't paying attention. I saw it, and know that you just smashed their food. you don't know that. If I come up to you, someone who cares about not smashing people's food for no reason, and say "I'm here to hold you accountable for what you've done wrong", you'll go "...what the heck?" but if I say "hey, you smashed their food," you'll turn around and be surprised and unhappy about this, not least because now you also know you need to clean up how it got on you, as well as the moral impacts. if you go around smashing people's food regularly, and repeatedly say you're just not paying attention, then it might be that me telling you you suck is an effective intervention, but it still seems unlikely; either you're doing it on purpose, in which case an immune response at you is needed, or you've got some sort of failure-to-update thing happening, in which case I need to help you look through what happens when you're about to make the mistake and find a new behavior there (a deeper version of explaining it). if it's because you're getting paid to not look around you on the subway, then it might be hard to get you to consider the tradeoff, but if you value not knocking people's food over, I might still be able to get through.

evhub's Shortform

13

Ω 10

evhub's Shortform

13

Ω 10