Thanks so much for the thoughtful response. I'll certainly reach out and try to participate in BlueDot Impact's course now that I'm more familiar with the content, and will stay on the lookout for anything you document as you go through your own journey! Even just a few of the names and resources so far have been incredibly valuable pointers to the right corners of the internet.
I don't have karma yet, but if I did, I'd gladly open my wallet :)
The timing of this post is quite serendipitous for me. Much of what you wrote resonates heavily. First comment on LW, by the way!
I'm deeply interested in the technical problems of alignment and have recently read through the AI Safety Fundamentals Course. I'm looking for any opportunity to discuss these ideas with others. I've been adjacent to the rationalist community for a few years (a few friends, EA, ACX, rationalism, etc.), but the need to sanity check my own thoughts on alignment has made engaging with the LW community seem invaluable.
I have found the barrier to entry seems high from a career perspective. Despite how I'd like to spend my time, my day job limits the amount of focused hours I can commit to upskilling in this domain, and a community of people in similar positions would be invaluable. I'm more than willing to self-study and do independent research, but I'm eager for some guidance so I can appropriately goal set.
If and only if the members have publicly provided the link, would you mind sharing any resources you find where groups may be working on particular problems?
Coauthor of the paper here. It's helpful to hear how people have interpreted our work, though I want to try to clear up a few things quickly.
Firstly, we don't claim that data must make up the remaining gap, and we certainly are not claiming it explains capabilities improvements more broadly. We try to be pretty precise about the definition of algorithmic progress under the CEG framework, which is specifically computed using loss rather than downstream performance metrics (I agree, the relationship between these is not straightforward or well understood). We say explicitly that the CEG framework cannot capture a lot of important innovations for performance and efficiency (instruction fine-tuning, constitutional AI, parallelism, etc.). We also show highly counterintuitive, undesirable properties of the CEG framework which makes interpretation quite difficult.
Speaking for myself and not my coauthors here: It's flattering you think our results are too good, though I'm not sure that was the intent of your comment :) I think the results would be interesting even if we over/undershot existing estimates by 10x. We find two important results, which are true regardless of our estimates' alignment with the literature: scale seems to be necessary for much of the perceived algorithmic gains, and scale-dependence makes the framework for measuring these gains behave strangely. Those two facts are worth considering in their own right, and are true for even modest differences in scaling exponents across architectures. It is reaffirming that we recover so much of the other estimates, but strictly speaking, we could have found way more or way less, and our results would still raise important points.
Lastly, I think our results don't point to "better data has made all the difference." It really points to "a lot of new, specific things lead to a lot of new, specific differences in capabilities, and it's hard to count those up using the same units." I think that's a really important (and difficult) direction for further research, which may shed light on how important data has been!