The key thing here seems to be the difference between understanding a value and having that value. Nothing about the fragile value claim or the Orthogonality thesis says that the main blocker is AI systems failing to understand human values. A superintelligent paperclip maximizer could know what I value and just not do it, the same way I can understand what the paperclipper values and choose to pursue my own values instead.
Your argument is for LLM's understanding human values, but that doesn't necessarily have anything to do with the values that they actually have. It seems likely that their actual values are something like "predict text accurately", and this requires understanding human values but not adopting them.
Inner alignment (mesa-optimizers) is still a big problem.
quick take: Roughly speaking adversarial examples are the Modern Reformulation you're asking about.
In my mind the main issue here is that we probably need extreme levels of robustness / OOD-catching. And these probably only come much too late, after less-cautious actors have deployed AI systems that induce lots of x-risk.
This is obviously vulnerable to adversarial examples or extreme OOD settings, but then robustness seems to be increasing with compute used, and we can do a decent job of OOD-catching.
This seems like the crux of the matter. I don't think OOD or robustness is as straightforward as you think.
The problem is how you incorporate that understanding into an optimization process, not necessarily how you get an AI to understand those values.
The best solution I can think of to outer-aligning an AGI capable of doing STEM research is to build one that's a value learner and an alignment researcher. Obviously for a value learner, doing alignment research is a convergent instrumental strategy: it wants to do whatever humans want, so it needs to better figure out what that is so it can do a better job. Then human values become an attractor.
However, to implement this strategy, you first need to build a value-learning AGI capable of doing STEM research (which obviously we don't yet know how to do) that is initially sufficiently aligned to human values that it starts off inside the basin of attraction. I.e. it needs a passable first guess at human values for it to improve upon: one that's sufficiently close that a) it doesn't kill us all in the meantime while its understanding of our values is converging, b) it understands that we want things from it like honesty, corrigibility, willingness to shut down, fairness and so forth, and c) that we can't give it a complete description of human values because we don't fully understand them ourselves.
Your suggestion of using something like an LLM to encode a representation of human values is exactly the lines that I think we should be thinking on for that "initial starting value" for human values for a value learning AGI. Indeed, there are already researchers building ethical question testing sets for LLMs.
This is obviously vulnerable to adversarial examples or extreme OOD settings, but then robustness seems to be increasing with compute used, and we can do a decent job of OOD-catching.
The issue is - as I understand it - under a sufficiently powerful optimizer "everything" essentially becomes adversarial, including OOD-catching itself.
It seems like we keep getting LLMs that are better and better at getting the point of fairly abstract concepts (e.g. understanding jokes). As compute increases and their performance improves, it seems increasingly likely that human “values” are within the class of not-that-heavily-finetuned LLMs.
For example, if I prompted a GPT-5 model fine-tuned on lots of moral opinions about stuff: “[details of world], would a human say that was a more beautiful world than today, and why?” I… don’t think it’d do terribly?
The same goes for e.g. how the AI would answer the trolley problem. I’d guess it’d look roughly like humans’ responses: messy, slightly different depending on the circumstance, but not genuinely orthogonal to most humans’ values.
This is obviously vulnerable to adversarial examples or extreme OOD settings, but then robustness seems to be increasing with compute used, and we can do a decent job of OOD-catching.
Is there a modern reformulation of “fragility of value” that addresses this obvious situational improvement? Because as of now, the pure "Fragility of Value" thesis seems a little absurd (though I’d still believe a weaker version).