ModelMirror

Regarding campaign spending and AI risk specifically, there is an interesting dynamic that I'd like to hypothesize could occur:

Let's imagine two coalitions. One is concerned with AI risk and is generally more "AGI-pilled" meaning that its members are more likely to believe that AI is going to be a huge deal, potentially the defining issue of the near-term future. The second sees AI as more of a "normal technology" and thinks that quickly advancing technology is usually very good for society. Let's assume that the AI risk coalition is largely correct, but ... (read more)

At a minimum, it seems like the lack of clarity/insight into the full process of certain results got produced makes it challenging to interpret model outputs. I commented here raising a similar point and referencing the OpenAI NS case.

I'm not particularly worried that a professional philosopher would hand over an essay for an AI to take credit

I agree, I meant my example above as an extreme hypothetical to demonstrate the issue, but I agree this is unlikely to actually happen for this competition.

this is partly because we don't have any level of assurance about almost anything about these systems!

I'm there with you on that. I think its worrisome that as systems have gotten more advanced it seems like the increased complexity of AI systems has implicitly been accepted as an excuse for lac... (read more)

I think this actually raises interesting question about how to interpret AI outputs. A similar issue seems to also have come up in the OpenAI Navier-Stokes kerfuffle. The initial statement from Tristan Buckmaster mentions this:

I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it eme