I have recently been very excited about Resolution, the new ASI alignment organization led by Geoffrey Irving. They seem to be making lots of good decisions.[1]Is any other major institution so directly addressing the “hard problem” of ASI alignment? They have already brought on great researchers and appear to have a high bar for applicants. I am particularly happy to see they have taken a multidisciplinary approach to AI alignment. This is great. Solutions could arrive from many fields. How cool is it that we have a well-funded agent foundations team again? Or that I could actually inform an epistemologist friend of a job opening in alignment research?
I also like that they’ve settled on a majority in-person approach, uniting a team in Berkeley. It’s a bit of a truism, but physical colocation helps a lot for fast-paced, nascent organizations. There are obvious advantages to sitting beside the major labs. However, I’m skeptical that the future of alignment research will depend on the work of a small number of teams in a single region. Shouldn’t we scale up research everywhere?
I’d love to see Resolution establish A(S)I alignment as an independent field of study that academic and non-academic researchers might conduct professionally and respectably at hubs around the world, whether that be at existing institutions, including universities, or new non-profits.
I am amazed by the number of academic researchers who have not switched to AI alignment research. Many fail to see the risks. But others must underestimate their ability to contribute, or have already invested a huge amount of career and social capital into their current position or field, or are settled enough that a relocation to a new city is hard. I would guess this is actually true for the majority of qualified researchers. Shouldn’t we find ways to facilitate their contributing to alignment?
Resolution looks well positioned to convince institutions globally that alignment is a serious new academic discipline. This could be a powerful multiplicative factor that exceeds what impact Resolution might achieve alone. Continuing to grow alignment organizations in the Bay Area would be great, but I think we are seriously missing out by not getting more existing institutions into the game.
My point would be made moot if automated alignment research allows us to concentrate the entire discipline into a single institution. Alignment might be solved via compute. However, I am skeptical this will happen in the short term.
So how might Resolution or an equivalent group help to grow alignment as a respected discipline? Well, the first is just by conducting excellent research beyond prosaic alignment. Having a serious, professional institution focused on ASI alignment already gives the field more credibility. But Resolution could also select research directions that will facilitate scaling up the field across numerous geographies and institutions. There are a lot of meta-questions that need answering. What is alignment research actually working towards? What might success look like? I have found that the best public resources for the alignment problem itself are often LessWrong posts.Published research meant to professionalize the parameters of the field, or to “prove” ASI misalignment is possible/likely/guaranteed, would help institutions to take alignment work more seriously. Generally, I would encourage some amount of public research that sends appropriate “social signals” to the existing research community. I haven’t thought deeply about this, though. Likely there are more and better approaches to consider.
All this would require careful management to ensure research directions are not narrowed prematurely for the sake of clean definitions, or that performative research is not conducted to gain credibility, and that none of this impedes their urgent core objectives.
I don’t have any contact with the Resolution team or know the details of their agenda. If you think my recommendation is ill-suited for their organization, or that their organization is actually “bad” in ways I don’t understand, fine! Then transfer my recommendation to another or yet-to-be-created organization working on ASI alignment. They should also help mature the discipline to allow for greater participation.
There is also a lot of work to be done here outside research itself. Media and professional publications would help. Perhaps outreach to key professors or university administrators, encouraging contributions outside computer science, or even creating new departments? Are there info hazard problems with conducting alignment research broadly? If so, can we still establish ethical standards for sharing work, or for what kind of research should be conducted? I doubt Resolution is equipped to do much of the possible field-building here. But it would be prudent of them to consider work that does build the field.
I know a coupleposts have already expressed concern about different research directions inside the organization. But most signals still look green to me! If you think I’ve misread the situation, please let me know.
I have recently been very excited about Resolution, the new ASI alignment organization led by Geoffrey Irving. They seem to be making lots of good decisions.[1] Is any other major institution so directly addressing the “hard problem” of ASI alignment? They have already brought on great researchers and appear to have a high bar for applicants. I am particularly happy to see they have taken a multidisciplinary approach to AI alignment. This is great. Solutions could arrive from many fields. How cool is it that we have a well-funded agent foundations team again? Or that I could actually inform an epistemologist friend of a job opening in alignment research?
I also like that they’ve settled on a majority in-person approach, uniting a team in Berkeley. It’s a bit of a truism, but physical colocation helps a lot for fast-paced, nascent organizations. There are obvious advantages to sitting beside the major labs. However, I’m skeptical that the future of alignment research will depend on the work of a small number of teams in a single region. Shouldn’t we scale up research everywhere?
I’d love to see Resolution establish A(S)I alignment as an independent field of study that academic and non-academic researchers might conduct professionally and respectably at hubs around the world, whether that be at existing institutions, including universities, or new non-profits.
I am amazed by the number of academic researchers who have not switched to AI alignment research. Many fail to see the risks. But others must underestimate their ability to contribute, or have already invested a huge amount of career and social capital into their current position or field, or are settled enough that a relocation to a new city is hard. I would guess this is actually true for the majority of qualified researchers. Shouldn’t we find ways to facilitate their contributing to alignment?
Resolution looks well positioned to convince institutions globally that alignment is a serious new academic discipline. This could be a powerful multiplicative factor that exceeds what impact Resolution might achieve alone. Continuing to grow alignment organizations in the Bay Area would be great, but I think we are seriously missing out by not getting more existing institutions into the game.
My point would be made moot if automated alignment research allows us to concentrate the entire discipline into a single institution. Alignment might be solved via compute. However, I am skeptical this will happen in the short term.
So how might Resolution or an equivalent group help to grow alignment as a respected discipline? Well, the first is just by conducting excellent research beyond prosaic alignment. Having a serious, professional institution focused on ASI alignment already gives the field more credibility. But Resolution could also select research directions that will facilitate scaling up the field across numerous geographies and institutions. There are a lot of meta-questions that need answering. What is alignment research actually working towards? What might success look like? I have found that the best public resources for the alignment problem itself are often LessWrong posts. Published research meant to professionalize the parameters of the field, or to “prove” ASI misalignment is possible/likely/guaranteed, would help institutions to take alignment work more seriously. Generally, I would encourage some amount of public research that sends appropriate “social signals” to the existing research community. I haven’t thought deeply about this, though. Likely there are more and better approaches to consider.
All this would require careful management to ensure research directions are not narrowed prematurely for the sake of clean definitions, or that performative research is not conducted to gain credibility, and that none of this impedes their urgent core objectives.
I don’t have any contact with the Resolution team or know the details of their agenda. If you think my recommendation is ill-suited for their organization, or that their organization is actually “bad” in ways I don’t understand, fine! Then transfer my recommendation to another or yet-to-be-created organization working on ASI alignment. They should also help mature the discipline to allow for greater participation.
There is also a lot of work to be done here outside research itself. Media and professional publications would help. Perhaps outreach to key professors or university administrators, encouraging contributions outside computer science, or even creating new departments? Are there info hazard problems with conducting alignment research broadly? If so, can we still establish ethical standards for sharing work, or for what kind of research should be conducted? I doubt Resolution is equipped to do much of the possible field-building here. But it would be prudent of them to consider work that does build the field.
I know a couple posts have already expressed concern about different research directions inside the organization. But most signals still look green to me! If you think I’ve misread the situation, please let me know.