James Butterworth

I have heard numerous claims recently that the underparameterisation of neural networks can be implied due to the polysemanticity of its neurons, which is prevalent in LLMs. Whilst I have no doubt that polysemanticity is the only solution to an underparameterised model, I urge on the side of caution when...