For the Astra Fellowship, what considerations do you think people should be thinking about when deciding to apply for SERI MATS, Astra Fellowship, or both? Why would someone prefer one over the other, given they're both happening at similar times?
Good question! In my opinion, the main differences between the programs are the advisors and the location. Astra and MATS share a few advisors, but most are different. Additionally, Astra will take place at the Constellation office, and fellows will have opportunities to talk with researchers that work regularly from Constellation.
MATS has the following features that might be worth considering:
Buck Shlegeris, Ethan Perez, Evan Hubinger, and Owain Evans are mentoring in both programs. The links show their MATS projects, "personal fit" for applicants, and (where applicable) applicant selection questions, designed to mimic the research experience.
Astra seems like an obviously better choice for applicants principally interested in:
Will more advisors be added later? The subareas covered by the advisors aren't as broad as I expected
We might add a few more advisors over the next few weeks, and there are a few advisors who chose not to be listed on the website. Are there specific subareas you'd like to see more of?
There's a sense in which there are specific assumptions made that influence the selection of advisors listed for Astra Fellowship. I may be wrong, but it seems to me that the majority of researchers listed seem to work on interpretability and evals-and-demonstrations, or have models of the alignment problem (or research taste and agenda) that are strongly Paul-Christiano-like.
I assume Chipmonk was gesturing at the nonexistence of advisors who aren't downstream of Paul Christiano's work and models and research agenda and mentors. Agent foundations (John Wentworth, Scott Garrabrant, Abram Demski) and formal world-models (Davidad) are two examples that come to mind.
Note I don't entirely share this belief (I notice that there are advisors who seem to be interested in s-risk focused research), but I get the sentiment. Also as far as I can tell, there are very few researchers like the ones I listed, and they may not be in a position to be an advisor for this program.
This is a link post for two AI safety programs we’ve just opened applications for: https://www.constellation.org/programs/astra-fellowship and https://www.constellation.org/programs/researcher-program
Constellation is a research center dedicated to safely navigating the development of transformative AI. We’ve previously helped run the ML for Alignment Bootcamp (MLAB) series and Redwood’s month-long research program on model internals (REMIX) in addition to a variety of other field-building programs & events.[1]
This winter, we are running two programs aimed at growing and supporting the ecosystem of people working on AI safety:
Applications for both are due November 10, 11:59pm anywhere on Earth. You can apply to the Astra Fellowship here and the Visiting Researcher Program here. If you are unsure about your fit, please err on the side of applying. We especially encourage women and underrepresented minorities to apply. You can refer others who you think might be a good fit through this form.
Logistics: Housing and travel expenses are covered for both programs, and Astra fellows will receive an additional monetary stipend. The start and end dates for both programs are flexible.
Questions? Email programs@constellation.org or ask them below.
Over 15 participants from these past programs are now working on AI safety at Anthropic, ARC Evals, ARC Theory, Google DeepMind, OpenAI, Open Philanthropy, and Redwood Research.