Hi! Equivarience is a very powerful tool and is widely applied in various branches of physics. Geometric Algebra Transformers generalize the concept of equivarience to transformers for essentially any group, e.g., $SO(3,1), SO(3)$ etc.
In fact, you can achieve drastic simplification and performance boosts by using equivariant models. An impressive example is in this paper: 19 Parameters Is All You Need: Tiny Neural Networks for Particle Physics (arXiv:2310.16121). An important point is that equivarient networks often significantly push capabilities.
A more general discussion on equivariant transformers is here: Geometric Algebra Transformer (arXiv:2305.18415).
Hi! Equivarience is a very powerful tool and is widely applied in various branches of physics. Geometric Algebra Transformers generalize the concept of equivarience to transformers for essentially any group, e.g., $SO(3,1), SO(3)$ etc.
In fact, you can achieve drastic simplification and performance boosts by using equivariant models. An impressive example is in this paper: 19 Parameters Is All You Need: Tiny Neural Networks for Particle Physics (arXiv:2310.16121). An important point is that equivarient networks often significantly push capabilities.
A more general discussion on equivariant transformers is here: Geometric Algebra Transformer (arXiv:2305.18415).