I wanted to share some reflections I have been having recently about how reinforcement learning in post-training may be affecting language models. This seems important for two reasons. First, much of the serious risk from advanced AI systems may come from post-training rather than pre-training alone. Second, reinforcement learning appears...
This is meant to be an introductory guide to anyone interested in the field of AI Safety. I want to give to you a general layout of this subject (part 1) and end with some practical steps that you can take if you wish to enter the field (part 2)....
Humanity is sitting on a volcano. Virtually out of nowhere, neural networks have expanded enormously in capability and generality (the power of exponential growth). As the models continue to improve month on month, many are asking, where is this headed? I, along with others, believe that this technology will lead...