Problem-wise, I'm very much with you.
Solution-wise, I have two concerns: timelines and effectiveness.
Timelines:
I don't think we have in any capacity close to the 2 years a program like this might take. Plus we need to set it up
first, and the first version of this is at most going to be half-good, and then we'd need to improve it, and the feedback loops are by default very, very long.
Effectiveness:
I'm very much unsure how effective university courses actually are compared to how effective they could be.
I'd guess it's less bad for newly created courses, but who knows.
Some pointers that they in fact are bad are:
- I've seen them not consistently employ worked examples for beginners
- They don't consistently employ practice tests for learning (and if they are, they're often graded, which makes them less effective)
- Spaced repetition is always left as an afterthought to be done by the students, which should already have a full mental workload if they're working hard. (And creating those cards on their own adds more mental workload to the students' lives which in effect should make them learn less)
Taking all of this together, to me it sounds quite a bit like we should probably hold off from the solution ('an uni course'), and think about the shape of the solution first.
So, what would be nice properties of the solution for this learning program?
It would be better if:
#0 It teaches the relevant stuff and is effective at that.
(Duh. But something that e.g. Bluedot doesn't necessarily ensure?)
#1 The feedback loops are fast, so that we could make it good fast.
So having many smaller programs/lecture bits would be better that a few big ones.
#2 It's incremental.
We already have a world full of experienced software engineers and academics that might want to contribute to this, and many of them already have a large chunk for the skills we need.
Having them go through only one small verification of their skills, and then some skill-up would mean way, way more poeple this could bring into the field
#3 It's verifiable from the outside.
So even though facilitated sessions are nice, it should be possible to see what the content of those sessions, and check that a person did the course in a sensible way.
Because that would mean more trust.
#4 (already partly point #2) It can take in an outrageous amount of people in a short time
I'm not sure how many people we need as staff for a METR-like org (probably not more than ~10 000?), but one very, very nice side effect of a program like this might be raising the sanity waterline in general, which would help us with policy stuff, which would be great.
(I'm sure there's more, but I have a very hard time not already thinking about a solution now)
To me, this sounds not very much uni-course-shaped, but very much BlueDot-shaped?
And even though BlueDot is in general doing a good job at getting people up-to-speed in AI Safety, they haven't been doing a particularly good job with regards to either teaching core EA principles, nor with teaching Rationality.
So maybe that is what we should be aiming towards?
I imagine this to be some kind of skill tree-shaped web platform, which you can work through in either of alone, in teams, or in teams with instructors that leads you towards being good at AI Safety. And this 'being good' would include all of the tech, rationality and EA basics that are not only necessary to contribute, but also sufficient to not fuck it up.
I'd need to think harder (so: spend some hours trying to actually design the course) for more better syllabus suggestions, but here goes:
Things that come to mind (all cut to the basics needed for each level, and as I said, I guess it should be possible to pick and choose):
- the karpathy building gpt-3 series
- basic linear algebra
- the transformer series of 3b1b
- overview of AI safety the academic field
- rationality basics
- alignment basics (like Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover)
- AI control & monitoring
- Cybersecurity basics
I share the skepticism re: "Is METR a Meaningful Check on Anthropic?" Let's take it as a given that we need an independent, government-funded agency involving thousands of independent auditors to pace and supervise the frontier AI labs. Let's even take it as a given that Congress will soon allocate, let's generously say, billions of dollars per year to this new agency. Let's imagine that the Hugging Face Incident, or some even more concerning incident yet to occur or be disclosed, ends up functioning as our new "Sputnik Moment" for AI Alignment against existential risk.
We would still have a problem: lack of qualified personnel with which to staff this new independent agency. I think we can all agree that just having a computer science degree does not really prepare someone for AI Alignment work, which is a pity because there are a lot of unemployed computer science majors out there. Like the US had to do to meet the 1950s Sputnik Moment, we would also need to overhaul the educational pipeline into this new field.
There are two ways I could see this being done:
#1: There are likely to be more applicants into this educational pipeline. With option #1, you'd be fishing among unemployed computer science grads, whereas with option #2 you'd be fishing among the entire college-attending population.
#2: I get the impression (correct me if I'm wrong) that a lot of computer science curriculum does not really improve someone's AI Alignment research usefulness. I mean, sure, a person needs to know what an "algorithm" is, and there are probably a lot of abstract concepts that are necessary, but my impression (as an outsider) is that a fair amount of a computer science degree will involve learning specific architectures and programming languages, which might not always be relevant to AI alignment research. Is there any way a 4-year degree could strip out the unnecessary specifics from a comp. sci. degree and add in the "Intellidynamics" stuff in their place?
#3: (This part is somewhat self-serving)...I enjoy the wishful thought that someone like me, an adjunct instructor in American history, a generalist who happens to already be familiar with the LessWrong corpus, the concepts that Liron Shapira groups under "Intellidynamics," a lot of relevant topics in philosophy, philosophy of science, economics, game theory, etc., could go from my current job (which is increasingly depressing in this age of AI cheating), to the point of being at least marginally-useful to an independent AI auditor agency (at least at a $30k/year entry-level position from which I can gain additional experience on the job), in a matter of 3 or 4 years rather than in 6 years after having to detour for a computer science degree first. I honestly don't think I have it in me to restart my career entirely like that, but if there were a way that I could start to be marginally useful to the field with about half that much preparation time, I would jump at the opportunity.
I suspect that many others in a similar position would be willing to enter the field as well if it were only, say, a 2-year program that one could add onto a previous bachelor's degree (i.e. a program focused entirely on the major itself, without the requirement to re-do all of the general education credits). The developed world is likely to see a glut of unemployed "public intellectual" types as AI automates more and more of education, instruction, etc., and it would be nice to find something for all of those people to do. My impression is that a lot of anti-AI discourse on the left-wing of the political spectrum, specifically the tendency to downplay AI progress as an overhyped bubble driven by corporate greed, comes from downwardly-mobile public intellectuals on the left who wishfully think that AI will not take their jobs, or should not take their jobs. If you want to see less of that sneering at AI existential risk concerns coming from that angle, it would be wise to find some way in which those types of people could be part of the solution rather than part of the problem.
Edit: An additional exercise to consider: what should be in a syllabus for a hypothetical "AI Alignment 101" or "Intellidynamics 101" course? The LessWrong sequences? What else? And who would be qualified to teach such courses? Basically, how would we bootstrap an entire field of AI Alignment research and supervision, if we all agree that there is a huge bottleneck of talent, and that qualified people coming out of computer science backgrounds tend to get snatched into much higher-paying AI capabilities jobs at the AI labs? How can we work around that dynamic?