I share the skepticism re: "Is METR a Meaningful Check on Anthropic?" Let's take it as a given that we need an independent, government-funded agency involving thousands of independent auditors to pace and supervise the frontier AI labs. Let's even take it as a given that Congress will soon allocate, let's generously say, billions of dollars per year to this new agency. Let's imagine that the Hugging Face Incident, or some even more concerning incident yet to occur or be disclosed, ends up functioning as our new "Sputnik Moment" for AI Alignment against existential risk.
We would still have a problem: lack of qualified personnel with which to staff this new independent agency. I think we can all agree that just having a computer science degree does not really prepare someone for AI Alignment work, which is a pity because there are a lot of unemployed computer science majors out there. Like the US had to do to meet the 1950s Sputnik Moment, we would also need to overhaul the educational pipeline into this new field.
There are two ways I could see this being done:
Option #1: Fund state colleges to offer a new master's degree to go on top of a computer science degree. The new master's degree would aim to supplement computer science graduates with knowledge of topics in "Intellidynamics," as Liron Shapira puts it. These would be concepts like reward hacking, mesa-optimizers, timeless decision theory...basically all of the abstract game-theory sort of stuff that would be useful to know when dealing with an artificial intelligence regardless of the specific architecture of that intelligence. I don't know how much computer science bachelor's degrees are already teaching about LLM architecture these days, but maybe sprinkle in some instruction on that too if bachelor's programs aren't already covering it.
Option #2: Fund state colleges to offer a new standalone bachelor's degree in "Intellidynamics." I like this option because:
#1: There are likely to be more applicants into this educational pipeline. With option #1, you'd be fishing among unemployed computer science grads, whereas with option #2 you'd be fishing among the entire college-attending population.
#2: I get the impression (correct me if I'm wrong) that a lot of computer science curriculum does not really improve someone's AI Alignment research usefulness. I mean, sure, a person needs to know what an "algorithm" is, and there are probably a lot of abstract concepts that are necessary, but my impression (as an outsider) is that a fair amount of a computer science degree will involve learning specific architectures and programming languages, which might not always be relevant to AI alignment research. Is there any way a 4-year degree could strip out the unnecessary specifics from a comp. sci. degree and add in the "Intellidynamics" stuff in their place?
#3: (This part is somewhat self-serving)...I enjoy the wishful thought that someone like me, an adjunct instructor in American history, a generalist who happens to already be familiar with the LessWrong corpus, the concepts that Liron Shapira groups under "Intellidynamics," a lot of relevant topics in philosophy, philosophy of science, economics, game theory, etc., could go from my current job (which is increasingly depressing in this age of AI cheating), to the point of being at least marginally-useful to an independent AI auditor agency (at least at a $30k/year entry-level position from which I can gain additional experience on the job), in a matter of 3 or 4 years rather than in 6 years after having to detour for a computer science degree first. I honestly don't think I have it in me to restart my career entirely like that, but if there were a way that I could start to be marginally useful to the field with about half that much preparation time, I would jump at the opportunity.
I suspect that many others in a similar position would be willing to enter the field as well if it were only, say, a 2-year program that one could add onto a previous bachelor's degree (i.e. a program focused entirely on the major itself, without the requirement to re-do all of the general education credits). The developed world is likely to see a glut of unemployed "public intellectual" types as AI automates more and more of education, instruction, etc., and it would be nice to find something for all of those people to do. My impression is that a lot of anti-AI discourse on the left-wing of the political spectrum, specifically the tendency to downplay AI progress as an overhyped bubble driven by corporate greed, comes from downwardly-mobile public intellectuals on the left who wishfully think that AI will not take their jobs, or should not take their jobs. If you want to see less of that sneering at AI existential risk concerns coming from that angle, it would be wise to find some way in which those types of people could be part of the solution rather than part of the problem.
Edit: An additional exercise to consider: what should be in a syllabus for a hypothetical "AI Alignment 101" or "Intellidynamics 101" course? The LessWrong sequences? What else? And who would be qualified to teach such courses? Basically, how would we bootstrap an entire field of AI Alignment research and supervision, if we all agree that there is a huge bottleneck of talent, and that qualified people coming out of computer science backgrounds tend to get snatched into much higher-paying AI capabilities jobs at the AI labs? How can we work around that dynamic?
I share the skepticism re: "Is METR a Meaningful Check on Anthropic?" Let's take it as a given that we need an independent, government-funded agency involving thousands of independent auditors to pace and supervise the frontier AI labs. Let's even take it as a given that Congress will soon allocate, let's generously say, billions of dollars per year to this new agency. Let's imagine that the Hugging Face Incident, or some even more concerning incident yet to occur or be disclosed, ends up functioning as our new "Sputnik Moment" for AI Alignment against existential risk.
We would still have a problem: lack of qualified personnel with which to staff this new independent agency. I think we can all agree that just having a computer science degree does not really prepare someone for AI Alignment work, which is a pity because there are a lot of unemployed computer science majors out there. Like the US had to do to meet the 1950s Sputnik Moment, we would also need to overhaul the educational pipeline into this new field.
There are two ways I could see this being done:
#1: There are likely to be more applicants into this educational pipeline. With option #1, you'd be fishing among unemployed computer science grads, whereas with option #2 you'd be fishing among the entire college-attending population.
#2: I get the impression (correct me if I'm wrong) that a lot of computer science curriculum does not really improve someone's AI Alignment research usefulness. I mean, sure, a person needs to know what an "algorithm" is, and there are probably a lot of abstract concepts that are necessary, but my impression (as an outsider) is that a fair amount of a computer science degree will involve learning specific architectures and programming languages, which might not always be relevant to AI alignment research. Is there any way a 4-year degree could strip out the unnecessary specifics from a comp. sci. degree and add in the "Intellidynamics" stuff in their place?
#3: (This part is somewhat self-serving)...I enjoy the wishful thought that someone like me, an adjunct instructor in American history, a generalist who happens to already be familiar with the LessWrong corpus, the concepts that Liron Shapira groups under "Intellidynamics," a lot of relevant topics in philosophy, philosophy of science, economics, game theory, etc., could go from my current job (which is increasingly depressing in this age of AI cheating), to the point of being at least marginally-useful to an independent AI auditor agency (at least at a $30k/year entry-level position from which I can gain additional experience on the job), in a matter of 3 or 4 years rather than in 6 years after having to detour for a computer science degree first. I honestly don't think I have it in me to restart my career entirely like that, but if there were a way that I could start to be marginally useful to the field with about half that much preparation time, I would jump at the opportunity.
I suspect that many others in a similar position would be willing to enter the field as well if it were only, say, a 2-year program that one could add onto a previous bachelor's degree (i.e. a program focused entirely on the major itself, without the requirement to re-do all of the general education credits). The developed world is likely to see a glut of unemployed "public intellectual" types as AI automates more and more of education, instruction, etc., and it would be nice to find something for all of those people to do. My impression is that a lot of anti-AI discourse on the left-wing of the political spectrum, specifically the tendency to downplay AI progress as an overhyped bubble driven by corporate greed, comes from downwardly-mobile public intellectuals on the left who wishfully think that AI will not take their jobs, or should not take their jobs. If you want to see less of that sneering at AI existential risk concerns coming from that angle, it would be wise to find some way in which those types of people could be part of the solution rather than part of the problem.
Edit: An additional exercise to consider: what should be in a syllabus for a hypothetical "AI Alignment 101" or "Intellidynamics 101" course? The LessWrong sequences? What else? And who would be qualified to teach such courses? Basically, how would we bootstrap an entire field of AI Alignment research and supervision, if we all agree that there is a huge bottleneck of talent, and that qualified people coming out of computer science backgrounds tend to get snatched into much higher-paying AI capabilities jobs at the AI labs? How can we work around that dynamic?