LESSWRONG
LW

emiddell
3010
Message
Dialogue
Subscribe

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
No wikitag contributions to display.
Mapping the semantic void: Strange goings-on in GPT embedding spaces
emiddell2y*41

In higher dimensions most of the volume of a n-sphere is found close to its shell.

The volume of such a sphere is Vn(R)∝Rn.  [1] 

The ratio of the volume of a shell to the rest of the ball is 

Vn(R+Δ)−Vn(R)Vn(R)=(1+ΔR)n−1 

which grows quickly with n. 

The embedding places the tokens in areas where there is the most space to accommodate them.

[1] https://en.wikipedia.org/wiki/Volume_of_an_n-ball

Reply
No posts to display.