Aumann in developed a formalism to reduce nashian game theory to bayesian decision making in a multi-agent setting and proved within that formalism that, under the conditions of common knowledge of "rationality" (read: common knowledge that every agent is running CDT) and that every agent knows his own action, the result will be a correlated equilibrium. As it turns out it is relatively straightforward to extend this framework to EDT and UDT like agents, which will be the goal of the rest of this post.
We will be using the same notation as Aumann: will stand for the space of all possible worlds (which we'll assume finite), p... (read 967 more words →)
The fact that language models inevitably will end up contradicting themselves is due to the fact that they have finite memory. Asking them not to contradict themselves over sufficiently large amount of text is asking for the impossible: they're figuring out what to output as the current token by looking at only the last n tokens, so if the contradicting fact lies further back than that there is no way for the model to update on that. And no increase in the size of the models, without fundamental architecture change, will fix that problem.
But architecture changes to deal with that problem might be coming...