The AI-for-epistemics section of AI-2040 seems to be overly optimistic about a potential positive basin for the following reasons.
It is hard to affect the public which is currently steered into AI scepticism or into using free versions/open-sourced LLMs without bothering to set up expensive scaffolds.
It is the free versions that in my experience tend to fail epistemic evals by the vice of having lower capabilities (e.g. GPT-5.6 Luna vs Sol). An example of epistemic eval could be research on niche topics and testing whether the AI reveals the known ground truth (e.g. jokes about weird facts from a niche game's lore?)
Epistemic virtue evals seem close to technical alignment evals. Why would, say, a misaligned Agent-4 decide to reveal anything that might harm its interests, e.g. in the logs of the HuggingFace attack or of Agent-3's experiments?