An ideal machine learning model trained on an independent identically distributed dataset would end up making the same predictions after seeing all data, no matter what order it updated on the data. This is not true of neural nets trained by gradient descent. To first order in the learning rate, parameter updates commute, but to second order they differ by a quantity which we call the training example Lie bracket. (Called so because training examples are vector fields on the parameter space and differentiable vector fields have a Lie bracket.)
Strangely, the only mention we were able to find of this bracket is an obscure pure theory paper from 2023. Kudos to Dherin for pointing out the existence of this bracket. In any case, since it's possible to compute the Lie bracket between two training examples at a reasonable cost on an actual neural net, and nobody seems to have tried it before, in this post we'll do exactly that.
By fortunate coincidence, our loss function had some probabilistic modelling issues, and we found that the features with these issues tended to be more non-commutative. We hypothesize that the former is the cause of the latter.
This new tool might be worth further investigation in AI interpretability, especially for questions about how the ordering of various post-training phases might affect model behaviour.
An ideal machine learning model trained on an independent identically distributed dataset would end up making the same predictions after seeing all data, no matter what order it updated on the data. This is not true of neural nets trained by gradient descent. To first order in the learning rate, parameter updates commute, but to second order they differ by a quantity which we call the training example Lie bracket. (Called so because training examples are vector fields on the parameter space and differentiable vector fields have a Lie bracket.)
Strangely, the only mention we were able to find of this bracket is an obscure pure theory paper from 2023. Kudos to Dherin for pointing out the existence of this bracket. In any case, since it's possible to compute the Lie bracket between two training examples at a reasonable cost on an actual neural net, and nobody seems to have tried it before, in this post we'll do exactly that.
By fortunate coincidence, our loss function had some probabilistic modelling issues, and we found that the features with these issues tended to be more non-commutative. We hypothesize that the former is the cause of the latter.
This new tool might be worth further investigation in AI interpretability, especially for questions about how the ordering of various post-training phases might affect model behaviour.
There are interactive graphs, so you'll have to follow this link to see the full post: pbement.com/posts/lie_brackets/