I'm very impressed by the proposed Total Research Transparency. I actually found it appealing even beyond the many reasons mentioned in the plan. It takes advantage of key properties of the current training paradigm, and this is actually desirable, because these properties are likely to remain in future AI systems:
Model training and serving will keep incentivizing a small number of large neural networks, with agent diversity coming from context. Efficiency of batched inference is too large, the learning algorithm and the hardware have co-evolved around it.
I'm very impressed by the proposed Total Research Transparency. I actually found it appealing even beyond the many reasons mentioned in the plan. It takes advantage of key properties of the current training paradigm, and this is actually desirable, because these properties are likely to remain in future AI systems:
- Model training and serving will keep incentivizing a small number of large neural networks, with agent diversity coming from context. Efficiency of batched inference is too large, the learning algorithm and the hardware have co-evolved around it.
... (read more)