I know this sounds fantastic but can someone please dumb down what KANs are for me, why they're so revolutionary (in practice, not in theory) that all the big labs would wanna switch to them?
Or is it the case that having MLPs is still a better thing for GPUs and in practice that will not change?
And how are KANs different from what SAEs attempt to do
- [4]
- AI life coaches
not excited about this - such a coach is either going to give very politically correct opinions, or target audiences with glaring insecurities, like young or low confidence men.. just like human coaches.
I dont know if you are aware, but this post was covered by Yannic Kilcher in his video "No, Anthropic's Claude 3 is NOT sentient" (link to timestamp)
If I understand this right, you train a transformer on data generated from a hidden markov process, of the form {0,1,R} and find that there is a mechanism for tracking when R occurs in the residual stream, as well as that the transformer learns the hidden markov process. is that correct?
Spot on