Here are some ideas for projects that people can do in AI Safety. It might be useful for you if you’d like to do something nice, but don’t know where to start or just generally looking for ideas. The list is going to be expanded and partially rewritten, but I believe it can be useful already.

Feel free to suggest entries or corrections in the comments!

Also, it would be cool if someone could help estimate the approximate average time for all links and how hard they are (I plan to do that later anyway)

I usually do not include here anything older than 2024, exceptions might be made for ‘previous versions’ or things that started a while ago, but frequentlty updated

Quotes from the links, explaining the essence of links are formatted as quotes

Research/pet projects lists

100+ concrete problems and open projects in evals

by Marius Hobbhahn

We made a long list of concrete projects and open problems in evals with 100+ suggestions! https://docs.google.com/document/d/1gi32-HZozxVimNg5Mhvk4CvW4zq8J12rGmK_j2zxNEg/edit?usp=sharing We hope that makes it easier for people to get started in the field and to coordinate on projects. Over the last 4 months, we collected contributions from 20+ experts in the field, including people who work at Apollo, METR, Redwood, RAND, AISIs, frontier labs, SecureBio, AI futures project, many academics, and independent researchers (suggestions have not necessarily been made in an official capacity). The doc has comment access, and further well-intentioned contributions are welcome!

Evals projects I'd like to see, and a call to apply to OP's evals RFP (2025)

by cb

I'm advertising this RFP in my professional capacity, but the project ideas below are my personal views: my colleagues may not endorse them.

There's just one week left before applications for our RFP on Improving Capability Evaluations close! The rest of this post is a call to apply, and some concrete suggestions of projects I'd be excited to see

Request for Proposals: Improving Capability Evaluations (2025) (they do not accept proposals anymore)
[ext] Projects someone should maybe do doc (2025)

Catherine Brewer

What this is:

We have an abundance of cyber evals and a lack of consensus on what results (if any) would be sufficiently scary to constitute a “red line”. (Also, we don’t agree on what such a red line would look like.)
As with bio red lines, I im

...

Posts

Wikitag Contributions

Comments