Thanks for the post! Not sure if I understood (I like it) it correctly about the idea of alignment between the geometry and the learned dictionary: this produces "separation" between features that is closer to the "true" dictionary—maybe it's just a paraphrasing of the same thing, but what did you mean by "semantic continuity"?
View as possible compression of human language? Can change my thoughts If they prove to have no consistency, or not interpretable/decodable by LLM.