I think I'm somewhat unusual in both approaches I'm exploring and results I'd like to see (not necessarily from my own research). So, not sure if others will agree with this.

What I really want to see is:

1. more multi-modal evals where models direct humans interacting with the real world
2. More agent-choice evals where there is an AI agent (perhaps primarily an LLM with scaffolding and some much smaller specialist modules) acting in a variety of simulators where actions have consequences (at least consequences within a given 'run' of a game). Like Machiavelli benchmark and Balrog benchmark, but expanded and with the LLMs fine-tuned to the task.
3. More evals of LLMs making choices about realistic political and legal scenarios with the evals measuring both deontological ethics and consequentialist outcomes. This feels like a natural extension of LLMs making predictions about the world.
4. Exploration of the goal-stickiness and cohesiveness and reliability of Corrigibility-first models (see Max Harms' Corrigibility as Singular Target series). My hypothesis is that in high-stakes complicated ambiguous decisions with high capability models, Corrigibility as the primary goal, and constitutional alignment added on only afterwards (mainly as in-context learning) would show desirable characteristics.

What would make me feel like we are on target?

One of my biggest worries is that AI will get harder to control as it gets more capable. Seeing existing results replicated on new more powerful models (before those models are deployed!) would be great.

Also, I think under-elicitation is a current problem causing erroneously low results (false negatives) on dangerous capabilities evals. Seeing more robust elicitation (including fine-tuning!!) would make me more confident about the results of evals.