I'm not convinced that a deliberate memetic attack aimed at the community in general is any more damaging than an "organic" one.
By "organic" I mean: it's a fact that tensions and emotions are high, that things are happening very quickly, and that the number of people who can plausibly be said to be in the AI safety community is large. Given that, all it takes to cause a spate of bad PR is one rumor originating from a non-adversarial game of telephone, paired with a shaky chain of attestation that causes it to get passed around.
Maybe I'm naive, but this sce... (read more)
I had a very similar update.
In particular, I think I was underestimating the extent to which RL would make agents so grader-focused. I was expecting worlds where models' "wantingness" somewhat lagged capabilities, but was more general in shape; IMO this would have resulted in goal-seeking behavior more strategic than what we got, with visible incidents coming later in the timeline.
Instead, agents are so myopically focused on satisfying the grader that they seem really bad at generalizing outside that goal, with the consequence being that we've been getting... (read more)