Thanks for the detailed post. I'd like to engage with one specific aspect - the assumptions about how RL might work with scaled LLMs. I've chosen to focus on LLM architectures since that allows more grounded discussion than novel architectures; I am in the "LLMs will likely scale to ASI" camp, but much of this also applies to new architectures, since a major lab can apply RL via LLM-oriented tools to them. (If a random guy develops sudden ASI in his basement via some novel architecture, I agree that that tends to end very poorly.)
The post series views RL as mathematically specified reward functions that are expressible in a few lines of... (read more)
Thanks for the detailed post. I'd like to engage with one specific aspect - the assumptions about how RL might work with scaled LLMs. I've chosen to focus on LLM architectures since that allows more grounded discussion than novel architectures; I am in the "LLMs will likely scale to ASI" camp, but much of this also applies to new architectures, since a major lab can apply RL via LLM-oriented tools to them. (If a random guy develops sudden ASI in his basement via some novel architecture, I agree that that tends to end very poorly.)
The post series views RL as mathematically specified reward functions that are expressible in a few lines of... (read more)