Nikita Kurdiukov

Likely, modern competitive Hebbian learning rules train Sparse Autoencoders implicitly - and their approximations make them worse at it. These algorithms specify weight updates directly, without computing a loss and the consequent gradients. But their updates contain terms closely related to both. This raises a question: how much of their...