Here's a thought I might expand on but that I'll just mention quickly now.
I feel there's a gap in the AI Safety field when it comes to automated alignment science.
There's a lot of talk about verification of individual knowledge claims and similar yet it doesn't feel like anyone is bringing principles from metascience and similar? If science is a collective process of generation and verification why are we not talking about things like generating distributed fault tolerance algorithms or ways of collectively verifying knowledge so that we know that any individual piece of work is right?
There's an existent field of meta science and whenever I see posts about automated alignment science I never see any discussion of this? Where are my Michael Nielsen references? Where are the ideas about proof burden and verification and thinking about this from a philosophy of science perspective?
See the following posts as examples, I don't see anyone citing or mentioning much metascience stuff. (I'm not saying these are bad posts, I want to point out that there's a missing mood):
Where exactly in science do we see a 1 on 1 verification regime where a singular human verifies the fact that something is true or false? Byzantine fault tolerance is a distributed scheme if you want convergent safety properties it makes more sense to look at systems with less individual failure points?
(Some stuff that examplify what I think should be looked at: )
I don't want to claim this with too much confident cynicism (also, I'm not really tracking the "automate alignment" literature much), but for the sake of completing the hypothesis space: this is roughly what you'd expect if large swaths of the discourse were not really serious about it and were largely sliding towards "automate alignment science" because it's a convenient cop-out for AI labs (and plausibly some other players) not having a good idea of how to make progress on this.
