Congrats! Could you say more about why you decided to add evaluations in particular as a new week?
It’s a fast-growing and important field right now - there is an urgency to make progress on eval, and a rapid increase in both technical safety eval roles at AI labs and governance roles. This need and capacity for safety evals make eval skills valuable for people who want to contribute to safety now. There are many methods that have been developed and relevant engineering skills to improve, but also a lot of minefields for producing false or misleading results. We thought the latter is an especially important reason for a good curriculum to exist
Asking for an acquaintance. If I know some graduate-level machine learning, and have read ~most of the recent mechanistic interpretability literature, and have made good progress understanding a small-ish neural network in the last few months.
Is ARENA for me, or will it teach things I mostly already know?
(I advised this person that they already have ARENA-graduate level, but I want to check in case I'm wrong.)
ARENA might end up teaching this person some mech-interp methods they haven't seen before, although it sounds like they would be more than capable of self-teaching any mech-interp. The other potential value-add for your acquaintance would be if they wanted to improve their RL or Evals skills, and have a week to conduct a capstone project with advisors. If they were mostly aiming to improve their mech-interp ability by doing ARENA, there would probably be better ways to spend their time.
ARENA has been successful because we had some of the best in the field TA-ing with us and consulting with us on curriculum design.
I personally love the ARENA curriculum, it has probably the single greatest resource that has helped me learning about current state of AI. I've also done couple of specializations on Coursera, but found the exercises lot easier - which also meant I didn't use everything I learnt in videos. On the contrary, all the ARENA exercises are challenging - but you also learn a lot more and it has definitely been more satisfying and rewarding journey for me.
Curious, if there are success stories to share from previous runs of ARENA? Example, maybe people publishing safety research or joining safety research labs.
Sorry for not seeing this. Hopefully, the first paragraph of the summary answers this question. We're excited about running more ARENA iterations exactly because its track record has been pretty strong.