Compression Regime Determines Whether a Neural Network Memorises or Discovers Concepts
The one-sentence version: A contrastive model trained on temporal co-occurrence in 10,000 novels, forced into a compression regime where it cannot memorise specific relationships, discovers transferable structural categories that correspond to narrative function rather than topic and these categories generalise to novels the model has never seen. The problem of...
Mar 211