Teaching LLMs to enjoy things. Some recent work into AI wellbeing has been pretty interesting. Language models represent emotion concepts linearly, they also do this for pleasure and pain, and even for reward and penalty. The thing that's obvious to me is that these are... in activation space! And there are a series of methods to make a model that's one way in one context the same way in another context (consistency training). Can we take tasks that models hate, and teach them to enjoy it, by regularizing their activations in training to conform to those on tasks they enjoy? And if so, how would we know if we're just smoothing over the model's verbalisation (but it's still suffering) or if we're actually helping? Perhaps through re-extracting such vectors, or probing with follow-up questions? How would such training affect downstream performance, persistence, tone, etc?

Another branch of this project would be to use these notions to construct things, rather than just measure things --- e.g., what's a training environment that a model would find fun, rather than just neutral or negative? And vice versa, what's an image that would cause a model to feel the most joy or suffering? Can we hill climb this?

All of this is related to AI welfare for obvious reasons, but they're also related to AI safety in that an AI that functionally suffers would probably hate us and be misaligned, and we may want to make deals or negotiations credibly with AIs, this would be one form of reward.

I will probably throw this at some SPAR mentees, but maybe other people are interested in working on this.