Researching energy-based models (inverse Predictive Coding) as an alternative to Deep Learning for LLMs, hoping that the local energy helps informs interpretability and monitorability.
I think we should distinguish between an idealized optimizer and the actual LLM agent, as IC manifests differently for each.
First consider a specific idealized optimal agent. Say it is a combination of an optimization module, and of a goal module. The goal module translates observations of the real world into a reward signal. For instance, a VLM or a CNN classifier answering "has coffee ?" every second, outputting reward 1 if it thinks so, 0 otherwise. The optimization module is software whose interface with the world is its action vector, its observation ... (read more)
I think we should distinguish between an idealized optimizer and the actual LLM agent, as IC manifests differently for each.
First consider a specific idealized optimal agent. Say it is a combination of an optimization module, and of a goal module. The goal module translates observations of the real world into a reward signal. For instance, a VLM or a CNN classifier answering "has coffee ?" every second, outputting reward 1 if it thinks so, 0 otherwise. The optimization module is software whose interface with the world is its action vector, its observation ... (read more)