About two months ago, someone asked me what I would do with more funding. Other than the obvious (i.e. generally improve my own quality-of-life in minor ways), my main answer was: take on an apprentice. I have some models about how best to train people for this sort of work, and an apprentice would allow me to test those models while also supporting my own research. I started laying groundwork for that plan - in particular, Specializing in Problems We Don’t Understand laid out my main background model.
Then, about a month ago, Aysajan put up a short post titled “Can I be Your Apprentice?” - essentially an open call to people on LW doing cool work. We talked, it seemed like a good fit, so the apprentice experiment kicked off ~3 weeks ago.
This post will provide more detail on models, motivation, the plan, etc, including a section for Aysajan to introduce himself.
First background model: Specializing in Problems We Don’t Understand. Problems-we-don’t-understand are similar to each other in a way which problems-we-do-understand are not. In the context of scientific research, preparadigmatic research in different fields is similar in a way which research within a paradigm is not. There are general skills and knowledge useful for finding/creating structure de novo, as opposed to working within some already-mapped structure.
Furthermore, while problems-we-don’t-understand may require some specialized knowledge, specialized knowledge of the field is never the rate-limiting step; if it were, then the problem would already be tractable to people steeped in the existing specialized knowledge of the field. If a problem is tractable within the current paradigm, then it isn’t preparadigmatic. Broad, generalizable skills/knowledge are much more important for problems-we-don’t-understand than for problems-we-do-understand.
The linked post goes into more detail on how one can train and specialize in problems-we-don’t-understand.
Second background model: Selection Has A Quality Ceiling. If we want people with a lot of skill in a lot of areas, trying to hire such people directly is Hard, in a big-O sense. As the number of traits we’re filtering for increases, the number of people we have to test in order to find one with all the requisite traits increases exponentially. The big-O requirements training skills are much better: as long as learning one skill doesn’t make another harder, the time required to train all of them should increase at-most linearly with the number skills.
Alas, most schools/companies today seem to mostly select, rather than train. Which makes sense - most companies don’t really need people with lots of skill in lots of areas, they just need people who will pick up the particulars of their industry quickly as-needed. But for problems-we-don’t-understand, people with lots of skill in lots of areas are exactly what we want.
Third background model: illegible skills. A lot of key skills/knowledge are hard to transmit by direct explanation. They’re not necessarily things which a teacher would even notice enough to consider important - just background skills or knowledge which is so ingrained that it becomes invisible. This sort of skill/knowledge is most easily transmitted by exposure: demonstration by the teacher, experimentation by the student, and feedback, ideally on a day-to-day basis. Thus the importance of an apprenticeship-like structure: high exposure and one-on-one interaction helps transmit illegible skills/knowledge.
(I suspect that this also relates to Bloom’s two-sigma problem: one-on-one tutoring works about two standard deviations better than anything else in education. Regardless of whether illegible skill transmission is actually a core part of that phenomenon, an apprenticeship certainly involves enough one-on-one tutoring that I expect the two-sigma benefit to kick in.)
Originally, I planned to put out a call for an apprentice around the end of this month/early next month. I hoped to get a few responses, filter for basic technical skills and personality compatibility, then randomly choose someone from a hopefully-not-too-short list. The intent was to avoid filtering heavily: I want to create new human capital, not merely select for existing human capital. And if the experiment works, I want to be able to do it again. Choosing someone who’s already obviously a uniquely good fit would compromise the information-value of the experiment.
Instead of that process, I’ve effectively selected on one thing: putting up a LessWrong post asking if anyone wants an apprentice. (Well, ok, I did also screen for basic technical skills and personality compatibility.) Aysajan’s resume is typical enough that I’m not too worried about selection effects there, but putting up a LessWrong post asking if anyone wants an apprentice implies a kind of chutzpah and do-what-it-takes attitude that may not be so easy to replicate. So from an experimental replicability standpoint, that’s a minor note of concern. (From a personal standpoint, I love it.)
From here, the plan is for Aysajan to spend the next few months working on the sorts of projects I worked on before focusing on alignment full time - the sorts of projects which I expect to build skills for solving problems-we-don’t-understand. These won’t be strictly or even primarily alignment-related; the goal is to build skill in solving problems-we-don’t-understand, and alignment is a pretty difficult area in which to practice.
Aysajan’s first post since the apprenticeship started went up just recently. It’s a write-up of an exercise looking for various systems besides probabilistic models which satisfy the assumptions used in Cox’ Theorem to derive Bayes’ Rule.
I don’t have any particular experimental outcomes to measure. If the project goes as well as I hope, then I expect it will be quite obvious. No need to go inventing proxies which don’t actually quite capture the things I care about.
(This section is in Aysajan’s voice.)
I am a business school faculty in a Canadian university. I earned my master’s degree in statistics and PhD in operations research in the US in 2018. I joined LessWrong not long ago and have been truly enjoying the intelligent conversations/debates going on here, whether it is related to general rationality, ML/AI, or simply investment. In the meantime, I find myself thinking Albert Einstein’s famous quote about learning quite frequently. He famously said “The more I learn, the more I realize how much I don’t know”. While being truly amazed by all the fascinating work LessWrong community members have been doing, I realize that I couldn’t do much due to my limited domain knowledge and limited hands-on experience. I am inspired and I want to contribute, especially in the fields of ML/AI research. But in reality, as an outlier in my current professional network (business academia), I am greatly struggling due to lack of guidance. Thus I made a call for an apprenticeship. I have a strong desire to contribute to the community by conducting original research and I believe apprenticeship is one of the best ways to learn ML/AI skills: learn from the best and do original research.
(This section is in John's voice, but speaks for both of us.)
Best-case scenario, this experiment provides a prototype for producing new specialists in problems-we-don’t-understand. At a personal level, we want to work with such people, and the ability to produce them would make that a lot easier. At a community level, alignment is a particularly difficult problem-we-don’t-understand (especially due to the lack of good feedback loops), and we hear that we’re now more bottlenecked on effective researchers than on funding.
But what we really want is a whole community or institute of people who specialize in problems-we-don’t-understand, making breakthroughs not only on alignment but on aging, on designing organisms from scratch, on efficient orbital delivery and terraforming, on generalized cognitive processes in organisms or organizations or brains, on fusion energy, on practical design of organizations or contract structures or memes, on cryptographically-secure biodefenses, …. We want a group of people who will set the whole damn world on fire.