Regarding OpenAI's most recent Prepardness Framework, I think it might fail for these reasons:

1. As an organisation, OpenAI seems to hold “learning by doing” as an institutional principle. In other words, models are evaluated partially by their real-world use and misuse after a process of iterative deployment. However, the structure of OpenAI and its incentives for deployment seem mismatched with safety goals. Even ignoring the fact that there is always a non-zero potential for harm from any deployed model (as the full extent of safety measures are explicitly framed as mitigations rather than “safety-first” ground-up designs), OpenAI has of course transitioned at some point during its existence to commercial enterprise chasing a profit incentive; disputes within the internal politics of the now closed-off organisation have definitely impacted the efficacy of safety efforts in its history, for example with the Superalignment team’s disbandment in 2024. Even in a technical sense, it is simply rational under our current economic system that such a company would prioritise profit as an instrumental goal to support its own existence. Further, structurally, the ultimate decision to deploy rests with the CEO of the corporation, which, in control theory, is definitionally a failed design (i.e. an interlock that can be bypassed by the system’s operator, who is incentivised to keep said system running, is fundamentally useless).
2. OpenAI relies on capabilities thresholds as benchmarks to evaluate risk and prioritise safety measures, essentially meaning that they will tend to examine a model’s empirical demonstrations of its performance rather than critically analysing and studying its actual cognition. This tendency leaves the overall framework, in my opinion, more vulnerable to problems such as emergent capabilities and even deceptive alignment. Both of these seem to somewhat be symptoms of the aforementioned “learning by doing” principle, whereby mitigations are designed as they are needed (i.e. arguably somewhat short-sightedly) rather than the cognition of the model itself being controlledly examined, critically and openly evaluated under diverse scrutiny (as this CSP research project intends to do). Although both emergent capabilities and deceptive alignment aren’t necessarily solvable by philosophically or mathematically pure means (one could, say, always just abstract to just another layer of “but what if the model isn’t ACTUALLY thinking what we think it’s thinking?”), a paradigm shift to a focus on something like “safe by design” mathematically provably models could be in order as these problems get potentially worse.
3. Lastly, OpenAI relies extensively on RLHF and human evaluations of model performance, which lends itself to problems like reward-hacking as well as in particular representing a *lack* of focus (an opportunity cost for resources, so to speak) on more fundamentally scalable and more automated systems of evaluation. Again, the organisation’s former Superalignment team had been somewhat focused on this problem (which does have the problem of a double-edged sword of capabilities externalities, though it is still an important avenue of exploration) by working on weak-to-strong generalisation and solving the so-called “novice vs. grandmaster” problem of evaluation, but disputes over compute and the company’s presumed prioritisation of profits over actual safety caused said team to be disbanded and its work to be supposedly “integrated” into that of OpenAI’s other teams.