This is a special post for quick takes by simulus. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
A recent paper probed LLMs and located both value features (representing the expected reward) and "dopamine" features (representing the reward prediction error). These features are embedded in sparse sets of neurons, and were found to be critical for reasoning performance.
Could these findings have any implications for model welfare?
If a model had mechanisms for "feeling good and bad", I imagine they would look similar to this.