This is a special post for quick takes by silentbob. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
The reason this visualisation is so persistent is that it taps into one of biological brains strongest inductive biases: stuff falls down.
A more useful (mental) visualisation is a 3D vector field with regions of low/high loss colorised as regions/clouds in space. This gives an extra degree of freedom back to weights.
It also enables you to think about things like weight decay as a gravitational pull towards the centre.