Road To AI Safety Excellence

Ruby (+32/-14)
Chris_Leong (+24/-2)
Arran_Stirton (-32) /* Task allocation */
toonalfrink (+155) deprecation
toonalfrink (-193)
toonalfrink (-43) /* External links */
toonalfrink /* Task allocation */
toonalfrink (+22/-57) /* Task allocation */
toonalfrink (+1736/-1577) /* Task allocation */
toonalfrink (+714/-1558) /* Course development process */

Road to AI Safety Excellence (RAISE), previously named AASAA, was an initiative from u/toonalfrink/toonalfrink to improve the pipeline for AI safety researchers, especially by creating an online course. See the Post-Mortem.

This page is deprecated, and it will no longer be updated by the RAISE founder (unless an independent party decides to). See the updated page at

Road to AI Safety Excellence (RAISE), previously named AASAA, iswas an initiative from u/toonalfrink/ to improve the pipeline for AI safety researchers, especially by creating an online course. See the Post-Mortem.


This page is deprecated, and it will no longer be updated (unless an independent party decides to). See the updated page at


This wiki hasn't been updated for a while. Some of its content is outdated. It will be updated one of these weeks (january 2018). In the meantime you can find updates in the Facebook group.

(edited as oflast updated at 2018-01-31, now roles have mostly crystallized)31)

Required technical understanding: none. none

(edited as of 2018-01-31, now roles have mostly crystallized)

Currently done by: Toon Alfrink, Veerle de Goederen, Remmelt Ellen, Johannes Heidecke, Mati Roy Required technical understanding: superficial. Minimum commitment: 21 full daysday per week

Currently done by: lots of people Minimum commitment: none

Instructional designContent developer ThereAs our content developer, you are many ways to get an idea across. There are documentaries, lectures, stories, texts, examples, proofs, assignments, etc. Different ideas need different media. It is hard to explain a math formula through a documentary,responsible for example. On the other hand, ifquality of the material. You coordinate the study group, review the quality of it's production, and spend extra time on your own learning the content (if you want to explain some philosophical idea like value misalignment, a made-up story like the paperclip maximizer would be a good fit. You decide the layout of a course unit. Ideally, there will be small juicy bits of information/questions/thingies in quick succession. A great example of a good course design is Google's deep learning course. Notice how most bits are only a few minutes long, and the student is prompted to think all the time. The course design donehaven't already) so faryou can be found hereour expert. You also help the lecturer with finalizing his scripts, and you assist him in understanding everything.

Currently done by: No one. This is a paid position. If interested, email us at raise@aisafety.campRequired technical understanding: moderate.near-complete. Minimum commitment: 12 full daydays per week

Giving lectures You thoroughly study the material, making sure you know it well enough to explain it clearly. Together with the instructional designer,content developer, you sit down and go over the bits that need explanation. These bits may range from 10 seconds3 to an hour of speaking,6 minutes, and they are interleaved with questions and small assignments to keep the student engaged. Then you teach the bits to a small class of testers, on camera. Testers will give immediate feedback: a few repeats may be necessary to get it right.

Currently done by: Robert Miles Required technical understanding: thorough. Minimum commitment: 1 full day per week

Writing materialStudy group attendant You write both assignmentshelp out in the weekly study group, creating summaries, mind maps, script drafts and text-based explanations, according to the request of the instructional designer. To what extent this is needed depends on the nature of the concept that is being covered.assignments. We also give presentations

Currently done by: Johannes Heidecke, Tom Rutten, Toon Alfrink, Tarn Somervell Fletcher, Nandi Schoots, Roland Pihlakas, Robert Miles, Rupert McCallum, Philine Widmer, Louie Terrill, Tim Bakker, Veerle de Goederen, Ofer Givoli Required technical understanding: thorough.none. Minimum commitment: 24 hours to 1 full day per week

Taking care of hostingSoftware developer WeWith about 60% certainty, we will host the course ourselves during development. Edxuse ihatestatistics as a platform. The company is open sourcerun by EA's (we may use it for free), and ideally it's specialization in statistics (which is closely related to AI) makes it well-suited for our needs. We have it set up using Amazon's EC2 hosting service. I am not evenHere is a linux noob (6 yearsdemo lesson. There are a lot of casual use), butdiamonds buried in the field of automated assessment. The quality of our answer-checking software determines the quality of the questions we can ask. Elaborate feedback mechanisms can make a lot of difference in how fast a learner may converge on the right kind of understanding. You write this took me a full weekend to set up. If you have system administration skills, you help is highly valuable for us.

Currently done by: Arran Stirton Minimum commitment: 1 hour2 days per week

Graphic design/animationAnimation & editing Good animation can make a course twice as polished and engaging, and this matters twice as much as you think. The whole point of a course instead of a loose collection of papers is that learners can trust they're on the right track. Virtue signalingPolish builds that trust. Animation is also a skill that is impossiblehard to pick up in a short enough timeframe.timeframe, so we can't do it. If you're interested in AI safety and skilled at animation, we really need you!

Now volunteers and capital are largely in place, we are doing an iterative development process with the first unit on corrigibility. When we are satisfied with the quality of this unit (which will probably take 1 or 2 more lecture meetings),unit, we will use the process we developed to create the other units.

Lecture meetings Every few weeks, a lecture meeting is held. A lecture room is set up with testers, lecturer and camera. By then, the instructional designer will have broken down the content into bits that are as small as possible. Best would be bits that can be explained in 3 to 6 minutes. For each bit, the lecturer explains it, and right after, the testers indicate how clear it was. If it wasn't clear enough, we repeat. At the first meeting it became apparant that repeating at least once is always a good idea, for the second try is always higher quality. Levels of understanding for the testers: 5. I would be comfortable explaining this in front of a class 4. I would be comfortable explaining this to a friend 3. I get the gist of it, but I would like to stare at this a bit longer 2. I have a superficial sense of what you probably mean 1. Wtf? The goal is to get everyone to at least level 3 from the lecture alone, and to at least level 4 when assignments are included.

- There are 'overview''scripting' and 'assignments' meetings.

- The 'overview' meeting asks it's attendants'scripting' meetings embody an iterative process to summarizego from papers to lecture scripts. We start with summaries, then we create mind maps, then we decide on a resource thatset of video, and then we create a set of script drafts based on summaries and mind maps

- All of this content is givenused by the lecturer to them. Then it asks every attendant to read every summary (or a subset of them). Then attendants are asked to construct a mind map (tree structure) offinalize scripts, set up the subject. Then attendants are paired up to cross-check their mind map with someone elsestudio and make amendments where they see

- The mind maps and summaries that areset of videos produced duringby the 'overview' meetinglecturer are used as an input to the lecture meetings.

- The set of lecture bits produced at the lecture meeting are used as an input to the assignmentassignments meeting.

Shooting lectures

We enlisted Rob Miles to shoot our lectures. About once a week, our content developer sits down with him to go over a particular script draft, which he modifies to his liking.

The setup includes a lightboard, which is a neat educational innovation that allows a lecturer to look at the camera while writing on a board simultaneously.

Load More (10/40)