The main reason being is that I think there's a very large correlation between "not being scary" and "being commercially viable", so I expect a lot of pressure for non-scary systems.
The scariness of RL systems like AlphaZero seems to go hand-in-hand with some really undesirable properties, such as [being a near-total black box] and [being incredibly hard to intentionally steer]. It's definitely possible that in the future some capabilities advancement might mean that scary systems have such a intelligence/capabilities advantage that this outweighs the disadvantages, but I see this as unlikely (though definitely a thing to worry about).
Curious what evidence makes you point towards "being a near-total black box" refrains adoption of... (read more)
For example, while inviting speakers was mainly Dmitrii's responsibility, it wasn't that easy and some of them were invited by me as I knew relevant people personally. We went over several smaller tasks together, like which questions to include in the feedback forms and what kind of information we needed to provide about Helsinki for people coming from abroad.
In addition, two dynamics really increased the number of our ad hoc meetings:
This post describes how we organized the Finnish Alignment Engineering Bootcamp, a 6-week technical AI safety bootcamp for 12 people. The bootcamp was created jointly with the Finnish Center for Safe AI (Tutke) and Effective Altruism (EA) Finland. It was composed of five weeks of remote learning based on the ARENA curriculum and a one-week on-site research sprint. We provide extensive details of our work and lessons learned along the way. Hopefully, this post helps others build similar programs and run them (or existing ones) more effectively. We don't focus here on our impact, although you can read about it here.
Thanks to
- Santeri Tani and Karla Still for their help with creating the program,
-... (read 4671 more words →)
Have to highlight here that this wasn't even the initial project of Mia and Zach; they pivoted halfway through the week from trying to craft better red team policies to this. Automated prompt engineering and fine-tuning from that seems to fit the blue team's bill better. The monitor as a classifier is arguably more straightforward than creating effective attack policies for the red team, although it's important to pursue strategies to find upper bounds for both teams.
Good post, thanks. I especially agree with the statements on current research teaching AIs synthetic facts and following through with deals during research.
It seems very likely that we have to coordinate with AIs in the future and we want maximum credibility to do so. If we aim to advance capabilities to program lies to AIs, while promising from safety research perspective, we probably should be very clear when and how this manipulation is done for it to not undermine credibility. If we develop such capabilities further we are also making the current most potential pathway to honor our commitments, fine-tuning the contract to the model, more uncertain from the model's perspective.
Committing to... (read more)
Exceptional work, well-founded and everything laid out clearly with crosslinks. Thanks a lot Steven!